•  Objective 1.2: Given a scenario, install, configure, and monitor kernel modules.

 A module (also called a kernel module) is a self-contained driver library file. The advantage of using modules, instead of compiling all their features into the kernel, is that they keep the Linux kernel lighter and more agile. We can add certain kernel functionality as needed or on demand because modules can be loaded and unloaded into the kernel dynamically. They extend the functionality of the kernel without the need to reboot the system.

We’ll take a look at the various kernel modules types, where their files are stored, and module configuration file locations. We’ll also explore in this chapter how to dynamically link and unlink the modules, view module information, and remove modules.

Exploring Kernel Modules

Kernel modules come in different flavors. They are as follows:

• Device driver: Facilitates communication with a hardware device.
• Filesystem driver: Required for filesystem IO.
• Network driver: Used to implement network protocols.
• System calls: Provides additional functions for adding/modifying system services.
• Executable loader: Allows additional executable formats to load.

There are a few different files and directories with which you should be familiar when working with modules. Modules required to support a kernel are stored in the /lib/modules/ directory tree. Each Linux kernel version available on your system has its own subdirectory within the /lib/modules/ directory. An example of this directory and its kernel version subdirectories on an Ubuntu distribution is shown in Listing 14.1.

Listing 14.1: Viewing a /lib/modules directory

$ ls -F /lib/modules
4.15.0-20-generic/  4.15.0-30-generic/  4.15.0-34-generic/
4.15.0-29-generic/  4.15.0-33-generic/  4.15.0-36-generic/
$

Be aware that distributions may implement this directory a little differently. For instance, the CentOS and Fedora distributions have the /lib/modules/ directory hard linked to the /usr/lib/modules/ directory. Thus, they are the same directory, but with two different names. An example on a CentOS system is shown snipped in Listing 14.2.

Listing 14.2: Viewing a /lib/modules and a /usr/lib/modules directory

$ ls /lib/modules
3.10.0-862.11.6.el7.x86_64  3.10.0-862.9.1.el7.x86_64 […]
$
$ ls /usr/lib/modules
3.10.0-862.11.6.el7.x86_64  3.10.0-862.9.1.el7.x86_64 […]
$
$ ls -id /lib/modules
63739 /lib/modules
$
$ ls -id /usr/lib/modules
63739 /usr/lib/modules
$

Notice in Listing 14.2 that the two directories share the same inode number. Hard links were originally covered in Chapter 3.

If needed, you can customize a module to define any unique parameters required, such as hardware settings essential for the device to operate. On some older Linux distributions there is a single configuration file, /etc/modules.conf, and on more modern distributions there are configuration directories:

• /etc/modprobe.d/ or /etc/modules-load.d/ contains configuration files generated at system installation or created by an administrator.
• /lib/modprobe.d/ stores configuration files generated by third-party software packages.
• /usr/lib/modprobe.d/, if it exists, is typically hard linked to the /lib/modprobe.d/ directory.
• /run/modprobe.d/ stores configuration files generated at runtime.

Within each configuration directory there are multiple configuration files that have a .conf file extension name. An example on a CentOS distribution is shown in Listing 14.3.

Listing 14.3: Viewing /etc/modprobe directories

$ ls /etc/modprobe.d
dccp-blacklist.conf     lockd.conf  truescale.conf
firewalld-sysctls.conf  mlx4.conf   tuned.conf
$
$ ls /lib/modprobe.d
dist-alsa.conf  dist-blacklist.conf  libmlx4.conf
$

Within the Linux virtual directory system there are many potential locations for kernel modules as well as module configuration files. Therefore, it is wise to take a look around your own Linux server and note their locations.

Many device driver kernel modules are loaded either at system boot time or dynamically when hardware devices are attached to the system. If problems occur, you need to know what utilities to use to help you diagnose the issue. There are three handy programs that can help with modules:

• dmesg displays the current kernel ring buffer.
• lsmod shows brief module information.
• modinfo provides detailed module data.

When it comes to modules, a module failure sometimes triggers a kernel message. The dmesg command is handy in that you can view kernel messages related to current events. At boot time, your distribution may take a snapshot of the kernel ring buffer and store the data in a file (typically the /var/log/dmesg file). Both of these ring buffer information sources can help you track down kernel module problems.

The dmesg utility will simply dump the current kernel ring buffer to STDOUT. It is helpful to employ the grep command to dig through the messages. A snipped example of this is shown in Listing 14.4.

Listing 14.4: Using dmesg with grep to display module messages

$ dmesg | grep -i driver
[…]
[    1.674321] e1000: Intel(R) PRO/1000 Network Driver - version 7.3.21-k8-NAPI
[    3.614747] cdrom: Uniform CD-ROM driver Revision: 3.20
[…]
[   48.828793] tun: Universal TUN/TAP device driver, 1.6
[ 8760.969714] usbcore: registered new interface driver usb-storage
[…]
$

You can employ different search terms to filter the dmesg utility’s output. For example, if you want to find information only on a USB device issue, you could pipe the dmesg output to the grep -i usb command.

The lsmod utility displays the status of modules currently within the Linux kernel. Listing 14.5 shows a snipped example of this on an openSUSE distribution.

Listing 14.5: Employing lsmod to display module status

$ lsmod
Module                  Size  Used by
af_packet              49152  4
[…]
bridge                172032  1 ebtable_broute
stp                    16384  1 bridge
btrfs                1327104  1
[…]
scsi_dh_alua           20480  0
$

In the lsmod output, each module is listed on a separate line. The first column is the module’s name.

Notice in Listing 14.5 the Used by column. The digit in this column indicates the number of processes or other modules currently using the module. If it is another kernel module using it, the other’s module’s name is displayed.

You can find out more detailed information concerning a particular kernel module via the modinfo utility. It may require super user privileges as shown snipped in Listing 14.6

Listing 14.6: Using modinfo to display detailed module information

$ sudo modinfo bridge
filename:       /lib/modules/[…]/kernel/net/bridge/bridge.ko
alias:          rtnl-link-bridge
version:        2.3
license:        GPL
suserelease:    openSUSE Leap 15.0
srcversion:     D39BA7E56E769F636E31A8C
depends:        stp,llc
retpoline:      Y
intree:         Y
vermagic:       […]SMP mod_unload modversions retpoline
$

Notice the kernel module’s file name in Listing 14.6. Kernel module files typically have a .ko file extension. Also notice that the module’s version number is displayed. This is helpful if you need to track down known bugs related to a particular module.

Installing Kernel Modules

Linux typically automatically loads modules as they are needed on the system. Often, though, you may want to test a new module or try new module configurations. To do so, you may need to manually install modules into the kernel. This is also called inserting or loading a module. In this section, we’ll look at a few utilities that can help you load modules into the kernel. They are as follows:

• insmod
• modprobe
• depmod

The insmod utility allows you to insert a single module into the Linux kernel. Unfortunately, because it is so basic, you have to provide an absolute directory reference to the module file. Also, the command does not load any needed module dependencies. A snipped example on an openSUSE distribution is shown in Listing 14.7.

Listing 14.7: Using insmod to insert a single module into the kernel

$ lsmod | grep -i joydev
$
$ sudo insmod  /lib/modules/[…]/kernel/drivers/input/joydev.ko
$
$ lsmod | grep -i joydev
joydev                 24576  0
$

In Listing 14.7, the system is checked for a loaded module that has a module name of joydev using the lsmod command. It is not found. Thus, the insmod command inserts it into the kernel using its full file name and directory location. The lsmod command is employed again to show that the module is now indeed loaded into the kernel.

The modprobe command is easier to use than the insmod utility because you can denote modules by their module name. It also loads any additional modules that the inserted module needs to operate (dependencies). A snipped example is shown in Listing 14.8, which employs the -v switch on the modprobe command to display more information while it inserts a module and all of its currently unloaded dependencies.

Listing 14.8: Using modprobe to insert a module and its dependencies

$ sudo modprobe -v dm_mirror
insmod /lib/modules/[…]/kernel/drivers/md/dm-log.ko
insmod /lib/modules/[…]/kernel/drivers/md/dm-region-hash.ko
insmod /lib/modules/[…]/kernel/drivers/md/dm-mirror.ko
$
$ lsmod | grep -i  dm_mirror
dm_mirror             28672  0
dm_region_hash        16384  1 dm_mirror
dm_log                16384  2 dm_mirror,dm_region_hash
dm_mod               139264  3 dm_mirror,dm_log,dm_multipath
$
$ sudo modinfo dm_mirror
filename:       /lib/modules/[…]/kernel/drivers/md/dm-mirror.ko
license:        GPL
author:         Joe Thornber
description:    device-mapper mirror target
suserelease:    openSUSE Leap 15.0
srcversion:     A784B0C071D49F47F94E83B
depends:        dm-region-hash,dm-mod,dm-log
retpoline:      Y
intree:         Y
vermagic:       […]SMP mod_unload modversions retpoline
parm:           raid1_resync_throttle:A percentage […]
$

The dm_mirror module allows volume managers to mirror logical volumes. In Listing 14.8 when the modprobe command is used to load this module, it loads two other modules as well, which the dm_mirror module needs to work properly. Notice that the modprobe utility is calling the insmod utility to perform the insertion work. Also notice that the dm_mirror module has a slightly different file name, dm-mirror.ko, shown in the modinfo utility’s output.

The modprobe program uses the modules.dep file to determine any module dependencies. This file is typically located in the /lib/modules/ subdirectory as shown snipped in Listing 14.9.

Listing 14.9: Viewing the modules.dep dependencies file

$ ls /lib/modules/[…]/modules.dep
/lib/modules/[…]/modules.dep
$
$ grep -i mirror /lib/modules/[…]/modules.dep
kernel/drivers/md/dm-mirror.ko:
kernel/drivers/md/dm-region-hash.ko
kernel/drivers/md/dm-log.ko
kernel/drivers/md/dm-mod.ko
$

In Listing 14.9, the modules.dep file is searched for the word mirror using the grep utility. To see a particular module’s dependencies, you locate the module’s file name within the file. After the colon (:), the module’s dependencies are listed by their full module file name. So for the dm_mirror modules (dm-mirror.ko), the module dependencies are the dm-region-hash, dm-log, and dm-mod modules. This corresponds with what was shown in the modinfo utility’s output in Listing 14.8.

You can employ the depmod command to scan through the system looking for any hardware that was not automatically detected. This is useful for troubleshooting problems with new devices. A snipped example is shown in Listing 14.10.

Listing 14.10: Using the depmod utility to update the modules.dep file

$ sudo depmod -v
[…]
/lib/modules/[…]/kernel/sound/soc/intel/
atom/snd-soc-sst-atom-hifi2-platform.ko needs
"snd_pcm_lib_preallocate_pages_for_all":
/lib/modules/[…]/kernel/sound/core/snd-pcm.ko
[…]
$

In Listing 14.10, the depmod utility scans the system, determines any needed modules, reviews the modules’ dependencies, and updates the appropriate modules.dep file. Notice it also displays its activity to STDOUT.

Removing Kernel Modules

It’s a good idea to remove any kernel modules you are no longer using on your Linux system. If you just need to remove (unload) a module with no dependencies, you can employ the rmmod command. An example is shown in Listing 14.11.

Listing 14.11: Using the rmmod utility to remove a module

$ lsmod | grep joydev
joydev                 24576  0
$
$ sudo rmmod -v joydev
$
$ lsmod | grep joydev
$

Notice in Listing 14.11 that the rmmod utility understands module names, so you don’t have to provide an absolute directory reference to the module file. Once the module is unloaded, the lsmod utility no longer displays the module’s name in its output.

The modprobe utility is useful for removing modules that have one or more dependencies. You just need to add the -r switch, and if you desire detailed information, include the -v switch, as shown snipped in Listing 14.12.

Listing 14.12: Using the modprobe utility to remove a module and its dependencies

$ sudo modprobe -rv dm_mirror
rmmod dm_mirror
rmmod dm_region_hash
rmmod dm_log
$

In Listing 14.12, the module dm_mirror is unloaded along with its two dependencies. Note that if the module was not loaded, you would not see any messages and just get a command-line prompt back.

Summary

When kernel modules fail or when you need to test a new module, it is vital to understand where module files as well as their configuration files are located. Equally important is the ability to diagnose problems using the various command-line utilities available for this purpose. Because modules can be dynamically linked and unlinked with the kernel, you should understand how to perform these tasks using the correct tools. These additional tools in your Linux tool belt will allow you to quickly resolve issues concerning kernel modules.

Exam Essentials

Describe the locations of kernel module files. Kernel modules files have a .ko file extension and are typically located in a subdirectory of the /lib/modules/ directory. There is a subdirectory for each particular Linux kernel version. Some distributions have additional directories, such as /usr/lib/modules/, which are hard linked to the /lib/modules/ directory.

Distinguish the locations of module configuration files. Older Linux distributions use a single file, /etc/modules.conf, as their kernel modules configuration file. More modern distributions use configuration directories, which can be the /etc/modprobe.d/, /etc /modules-load.d/, /lib/modprobe.d/, /usr/lib/modprobe.d/, and/or /run /modprobe.d/ directory. Within configuration directories, module configuration files have a .conf file extension.

Summarize the utilities to troubleshoot modules. Because when kernel modules fail, they often issue a kernel message, you can employ the dmesg utility to view recent kernel messages or peruse the /var/log/dmesg file, if available, for boot time kernel problems. The lsmod utility displays all the currently loaded modules, the number of processes and other modules using them, and the other module’s names. The modinfo program is very helpful because it displays detailed information concerning a module, including its dependencies.

Compare the utilities to install kernel modules. The low-level insmod utility requires a full module file name in order to insert a module into the kernel, which can be cumbersome. In addition, it does not load any module dependencies. On the other hand, the modprobe utility only requires the module’s name. Also, it searches the modules.dep file to determine and load any module dependencies.

Explain the utilities to remove kernel modules. The rmmod utility is a low-level utility. Though it does not require a full module file name in order to unlink a module from the kernel, it does not unload any module dependencies. So you could end up with unneeded modules, still linked to the kernel. The modprobe utility, using the -r option, will unload the module and unlink any module dependencies.