Working with the Kernel
The heart of your computer is the kernel. This kernel works with specific parameters and modules, and as a Linux user you need to have at least a minimal knowledge about them. In this chapter, you’ll learn how to perform basic kernel management tasks and how to change parameters for your kernel. You’ll also learn how to configure GRUB to load your kernel.
Understanding the Kernel
As mentioned, the Linux kernel is the heart of the operating system. It is the software that communicates directly to the hardware. The kernel is the only part of the operating system that communicates to the hardware directly; all other components that you use have to go through the kernel as shown in Figure 13-1.
Figure 13-1. Only through the kernel can all software communicate to the computer hardware
To access the different hardware components, the kernel needs drivers. Every kernel driver is represented by a kernel module. Only the most essential drivers are directly compiled in the kernel itself. While booting, the kernel loads in initial RAM disk. This RAM disk contains drivers for hardware that may be present in your computer. These drivers are not included in the kernel itself, because these are non-essential drivers.
In general, drivers are loaded automatically when the computer boots through the kernel directly or by using the initial RAM disk or when new hardware is attached to the computer. The latter is done with the aid of the udev process, which is used by all modern distributions.
Note In the old days, kernels were monolithic, meaning that all drivers were compiled directly into the kernel. This also meant that if a user needed a new driver, he or she had to enable the driver in the source code and recompile the entire kernel. Fortunately, this is no longer necessary; if a new driver is required, the user just has to load a new module. Therefore, situations where kernels need to be recompiled are pretty rare nowadays.
Managing Kernel Modules
To work with your computer’s hardware, you need drivers to access the hardware. The role of the driver is to tell the kernel exactly how it should address the hardware. In general, you don’t need to do anything for proper hardware access, but some cases will require your involvement, and Linux offers some commands to help you with that. These commands are discussed in the following sections.
Before doing any module management on your computer, you should know which modules are loaded. For this purpose, you can use the lsmod command. Listing 13-1 shows you the command and sample output.
The output of lsmod shows you not only which modules are loaded, but also what the modules are doing. In the first column of its output, you see the name of the module (for instance, on the last line, you see information about ide_core). Next is the amount of memory the module uses (117648 in the case of usb_core), followed by the number and names of the other modules that currently use this module. In particular, the latter part of this information is important to know about, because if a module has dependencies, you cannot just unload it before unloading the dependencies. Before doing anything, you should use lsmod to see what the modules are currently doing, after which you can use modprobe or modinfo on these modules.
Loading and Unloading Modules with modprobe
Basically, modules get loaded automatically. In some cases, however, you may need to unload and reload a module by hand. This may be required after changing options for a module (see the section “Changing Module Options” later in this chapter for more information). As modules normally get loaded automatically, you’ll probably have to unload a module first before you can load it again. To unload a module that is currently loaded, use modprobe -r. For example, the following command would unload the fat32 module from your computer:
modprobe -r fat32
modprobe will not normally return any messages; it just does its job and quits. If you actually want to see what it is doing, you can add the -v option for verbosity. This can be especially useful when loading a module; for instance, it shows you what dependencies are automatically loaded with your module. Listing 13-2 gives an example of this. Notice that modules that are currently required by another kernel module as a dependency cannot be unloaded.
Loading of a module may fail because of versioning errors. Typically, modules are written for a specific version of the kernel, and if the version in the module is wrong, it will fail to load. A bad solution to this problem is to disable version checking by using the -f (force) option. Notice that this should never happen for kernel modules that where installed with the kernel. Versioning problems can occur though if an administrator has manually installed Linux kernel modules that are not a part of the official Linux kernel, the proprietary kernel modules.
You may succeed in loading the module using the option -f, but it won’t be stable. However, if this is your only option, you may need to use it. Before using modprobe -f, you should first check whether you can find the source code for your module, which you might want to compile again to make sure it works with your kernel. See the section “Compiling Modules” later in this chapter for more information.
Displaying Module Properties with modinfo
In some situations, you may just want to know more about a module that is currently loaded. To do this, you can use modinfo. This is a pretty straightforward command that gives you all available properties for a module, which may be useful for troubleshooting. For instance, modinfo also gives you the license that is used for the module in question. This allows you to recognize a proprietary module, which may possibly cause problems in your current kernel. In Listing 13-3, you can see the result of running the modinfo command on the vfat module.
Changing Module Options
When working with kernel modules, you can pass options to them. You would do this by editing the /etc/modprobe.conf configuration file or by adding module-specific files to /etc/modprobe.d. You can include lines for specific modules in modprobe.conf and specify all you need to do on one line per module, or alternatively you can create a configuration file for a specific module in /etc/modprobe.d. Note that the letter currently is the preferred method for loading kernel modules. Listing 13-4 shows partial sample contents of modprobe.conf.
As you can see, a few commands are used in modprobe.conf or the files in /etc/modprobe.d:
- alias: Use this to give an alternative name to a module. In the example line alias personality-14 abi-uw7, the alias name personality-14 is given to the real module name abi-uw7. This means that you can also load this module by referring to personality-14.
- options: This command is used to pass specific options to a module. In the example line options sisfb mode=none mem=12288, no specific mode is specified, and a memory address is added. Module options are normally module specific and in some cases are required to get the appropriate behavior from a module. You can find out which module options exist for a specific module by using the modinfo command. The last lines of the output of this command show which options are supported (if any) by lines that start with the text parm.
- install: Normally, a module is just added to the kernel. If your module needs more parameters and settings when it initializes, you can use the install command, which lets you use complete shell scripts to load a module. An example of this is in the line install ata_piix /sbin/modprobe ahci 2>&1 |:; /sbin/modprobe --ignore-install ata_piix, which tells modprobe that it should load the ata_piix module by doing a modprobe on the ahci module; if that fails, it should not load ata_piix at all. (More on the techniques that are used in a shell script like this will be discussed in the next chapter.)
- remove: Like install, this command allows you to pass specific options when unload- ing a module.
- include: By using the include command, you can tell modprobe to use an additional configuration file for loading modules. As you can see in Listing 13-4, the include command is used to include all configuration files in /etc/modprobe.d as well.
- blacklist: In some cases, a module is programmed to use internal alias names. These may conflict with an alias name that you have configured for the module. To prevent problems with this, you can use the blacklist command to indicate that all of the module’s internal alias names should be ignored. Some distributions have an /etc/modprobe.d/blacklist configuration file in place by default to prevent certain alias names from being used.
As you can see, when using the lsmod command , some modules depend on other modules to load successfully. Scanning for these module dependencies is the responsibility of the depmod command, which automatically loads when your computer boots. The result of this command is written to the modules.dep file, which you can find in the modules directory for your current kernel (/lib/modules/...). Notice that normally you should not need to manually use the depmod command as this command is automatically executed when the kernel is installed, or when kernel updates are installed.
Since this file just contains a long list of modules and their depen- dencies, it makes no sense to edit it yourself. In case a module has dependency problems, it can be useful to run the depmod command again from the command line. This command will generate a new modules.dep file automatically for you.
Legacy Commands for Module Management
Some older commands for module management don’t take module dependencies in consid- eration. You should not use these commands, which I’ve listed here, because they most likely won’t load and unload your modules correctly:
- insmod: Loads a module
- rmmod: Removes a module
EXERCISE 13-1: WORKING WITH KERNEL MODULES
- Type lsmod to get an overview of currently loaded kernel modules. Check if the vfat module is loaded using lsmod | grep vfat. The next steps in this exercise assume that it has not been loaded.
- Type modprobe vfat to manually load the vfat module.
- Repeat the command lsmod | grep vfat. You will now see it loaded, and you’ll also see that it has a dependency to the fat kernel module.
- Type modinfo vfat. You won’t see any kernel module options near the end of the result of this command.
- Type modinfo cdrom. Notice the parm lines near the end of the output, which indicates the module options that can be used.
- Type modprobe -r vfat. This unloads the vfat module.
- Type lsmod | grep fat. Notice that not only the vfat module, but the fat module as well have been unloaded.
Finding the right Hardware Module
For hardware on your computer, the right modules should be loaded automatically. You can verify if this is the case using the lspci -k command. Listing 13-5 shows partial output of this command, notice the last line of every PCI address that mentions the name of the PCI module that is loaded.
In some cases you won’t see any Kernel driver in use line. If this is the case, no kernel module could be found for this specific hardware. To fix this issue, you should try to find a proprietary driver with the vendor of the specific hardware.
Tuning Kernel Parameters
In the old days, tuning kernel parameters was hard. You needed to change parameters in the C language source files and recompile the kernel to make changes. Today this is no longer a requirement. Instead, you can write new parameters to the /proc file system. This file system contains several kernel settings, and by writing directly to some of the files in /proc, you will immediately change the setting.
You should never try to change settings in /proc without knowing what you are doing, because you may severely trash your system. However, /proc expertise requires deep insight into the working of Linux and the Linux kernel; hence, I won’t give you a complete list of every parameter that you can change in /proc. I will give you some useful examples, however, as well as the method to make changes permanent.
Writing Changes to /proc
To write a change to a file in /proc, you need to echo the new value to the configuration file. All kernel-related configuration files are in /proc/sys. Let’s consider an example: the con- figuration file /proc/sys/net/ipv4/ip_forward indicates whether your computer can route packets between two network cards. Typically, this is not required for an end-user compu- ter, but you may choose to set this up if you want to use your computer as a wireless access point, for instance. You can show the default setting in this file by using cat /proc/sys/net/ ipv4/ip_forward, which will give you the value of 0, meaning that routing currently is disa- bled. To enable it, echo the value of 1 to this configuration file. Listing 13-6 shows how this procedure works.
The disadvantage of the procedure just described is that changes are not persistent when you reboot your computer. This means that after a reboot, you would have to apply all of these settings again. Fortunately, a workaround exists in the form of the sysctl package. If installed, sysctl runs as a service when your computer boots. When loaded, it reads its configuration file, /etc/sysctl.conf. This file contains a list of all parameters that have to be applied to the /proc file system. Listing 13-7 shows an example of what the contents of sysctl may look like. This example, which comes from an Ubuntu server, contains some valuable information on param- eters that you may want to change.
In /etc/sysctl.conf, configuration files in /proc/sys are referred to by using relative path names. As you can see in the last line, for instance, the setting net/ipv4/ip_always_defrag is used, which refers to the file with the complete name of /proc/sys/net/ipv4/ip_always_ defrag. Instead of using the notation with slashes that you see in this example listing, your distribution may use the dotted notation, which would in this case be net.ipv4.ip_always_ defrag. It doesn’t matter which you choose, as both are compatible.
When changing kernel settings, you should know where to find which kind of settings. In /proc/sys, different subdirectories are used to group different kinds of configuration files. The following subdirectories are used:
- debug: Contains parameters related to kernel debugging
- dev: Contains parameters related to devices and their working
- fs: Contains file system parameters
- kernel: Contains kernel-related parameters
- net: Contains network-related settings
- vm: Contains settings that relate to memory management
Some Useful /proc Parameters
As mentioned before, tuning the kernel by modifying the /proc file system is not easy to do and requires deep insight in the working of the kernel. Therefore, I will not go into that here. To give you an impression of the possibilities, I’ve included a short list of some of the options in /proc:
- /proc/sys/net/ipv4/ip_forward: Determines whether your computer has to route packets between network cards. Use this if you are setting up your computer as a router on a network.
- /proc/sys/dev/cdrom/autoeject: Tells the kernel whether it should automatically eject the optical disk after unmounting it.
- /proc/sys/dev/scsi/logging_level: Determines the logging level that SCSI devices should use. A higher log level means more intensive logging. Only use values here that are the double or the half of the previous value; for example, 1, 2, 4, 8, 16, 32, 64, 128, 256. Tune this if you want more (or less) SCSI-related logging.
- /proc/sys/fs/file-max: Gives the maximum number of files that can be opened simultaneously.
- /proc/sys/kernel/hostname: Contains the name of the computer as it is known by your kernel.
- /proc/sys/kernel/osrelease: Contains the current kernel version. This file is read when displaying the current version on the command line with the uname -r command.
- /proc/sys/net/core/rmem_max: Sets the maximum amount of memory that the kernel should reserve to buffer incoming network packets.
- /proc/sys/vm/laptop_mode: Tells the kernel whether it should run in laptop mode. When enabled with the value 1, it will use settings that are more energy efficient.
- /proc/sys/vm/swappiness: Tells the kernel how fast it should start swapping. A higher value in here indicates a higher willingness on the part of your kernel to start swapping.
EXERCISE 13-3: MAKING KERNEL CHANGES THROUGH /PROC
- Open a root shell.
- Type sysctl -a | grep icmp. This shows all current ICMP related proc settings. Notice that the parameter icmp_echo_ignore_all should be set to 0.
- Type ping localhost. You are getting a reply because ICMP traffic is allowed.
- Type echo 1 > /proc/sys/net/ipv4/icmp_echo_ignore_all
- Repeat step 3. You should no longer be able to ping localhost.
Compiling Your Own Kernel and Kernel Modules
Among the advantages of using Linux is that you can create new functionality based on the source files of kernel as well as other programs. In this section, you’ll read how compiling is used to get things going, first as it applies to the kernel, and second as it applies to new kernel modules.
To be able to build and compile your own software, you must have a C compiler installed on your computer. Typically, the GNU C compiler, which is in the gcc package, is used for this purpose. Make sure that it is installed, as well as all the packages that depend on it, before pro- ceeding. Read Chapter 8 for more information about the installation of new software packages on your distribution.
Note: On enterprise Linux distributions such as SUSE and Red Hat, it is not common to compile your own kernel. The kernel source files often aren’t even included. This is because for those distributions the supportability is very important. By recompiling your own kernel, you’ll break support as you are going to include options that haven’t been verified by the vendor. So if you’re using an enterprise Linux distribution, don’t compile changes into your kernel.
Understanding Make
To compile software, you need a C compiler, which is in the gcc package. To compile C files successfully, generically speaking you’ll use one of two approaches; the one you use depends on which way the source program files have been delivered. If your source file is one file only, you could compile it directly, using the gcc command. When dealing with Linux kernels and drivers, this never is the case; you don’t work with one single source file, but with lots of source files that are all linked to each other and, depending on your current configuration, have to behave differently. As it would be virtually impossible to compile all of these one by one, most source files come with a file that helps with the compilation process, named Makefile. You will find it after extracting the source files from the software package.
The Makefile contains specific instructions that tell the C compiler exactly what it has to do when compiling the software. To start compiling the software against this Makefile is not too hard: you just run the make command. However, since there can be different Makefiles on your computer, you should always run the make command from the directory that contains the Makefile you need to start compiling your software.
If the software in question is very complex, the Makefile may contain instructions to run the compiling job for different scenarios. This is the case when working with the kernel, where you can run make with different arguments to tell the compiler exactly what it has to do. These arguments are not specific to make but are defined in the Makefile itself. If you’re not afraid of complicated scripts, you can even read the Makefile to try to find out exactly what it accomplishes. (See http://www.gnu.org/software/make/manual/make.html for more information on make.)
Modifying and Compiling the Kernel
Before the Linux kernel was modular and before it was easy to write new kernel options to the /etc/sysctl.conf file to change settings dynamically, you needed to change kernel settings and recompile a new kernel after making your setting changes. Nowadays, there is hardly a reason to proceed in this way. Nevertheless, this section gives some insight into what is needed to configure and compile your own kernel, which will result in a new kernel. Before you start, make sure that the kernel source files are installed on your computer. Read Chapter 8 for more details on how to install software.
It only makes sense to compile a kernel if you have changed settings to the kernel. The next section shows you how to do so, and the section after that discusses how to compile the kernel after making these changes.
Modifying the Kernel
The current kernel configuration is normally stored in the .conf file in the directory /usr/ src/linux, which contains the source files of your current kernel. To create this file, different options are available, of which two will be covered here:
- Create the .conf file from scratch.
- Create the .conf file based on the default configuration.
Creating a .config file from Scratch
To create a .config file from scratch, you need to run the make config command or one of the related commands. This command starts a script that asks for every single piece of kernel functionality how you want to configure it, which is very user unfriendly. Some alternatives exist that are easier to use: make menuconfig and make xconfig. Both offer a menu interface that enables you to specify what you need in your new kernel configuration. As they offer the same functionality, I will cover the make menuconfig command only; you can run it from the console, which is not the case for make xconfig.
You must run the make config commands from the /usr/src/linux directory, so before you start, make sure that you are in this directory. Next, type make menuconfig to start the menu-based kernel configuration. Figure 13-2 shows what the interface looks like.
Figure 13-2. make menuconfig offers a menu interface that allows you to change kernel options
In the make menuconfig interface, all options are subdivided in different modules that allow you to find specific functionality easily. You can select an option by navigating to it with the arrow keys and pressing Enter to select it. Some options can just be switched on or off, whereas other options can be enabled as a module as well. For the latter option type, you can toggle between selected, unselected, or an M, which indicates that you want to use the option as a module (see Figure 13-3). You can also get more details about what functionality is related to a given option: select it first and then use the Tab key to navigate to the Help option. Select- ing Help will show you a description of the selected function.
Figure 13-3. From make menuconfig, it is easy to switch options on or off
Once you are finished creating the configuration you need with make menuconfig, you can browse back to the top of the menu interface by selecting Exit until you are back in the main menu. This offers you the screen shown previously in Figure 13-2. From there, select Exit once more. You will now be prompted to save the current selection. Select Yes to write the current selection to a configuration file. Read the section “Compiling a New Kernel” later in this chap- ter for information on how to proceed from this point.
Creating a .config Based on the Default Configuration
Every kernel also contains a default configuration, which according to the kernel maintainer contains the best options for the architecture that you are using. You can easily write this default configuration to the /usr/src/linux/.config file by using the make defconfig com- mand. Make sure that you use this command from the /usr/src directory, as it otherwise won’t work.
After starting the make defconfig command, lots of options will scroll over your screen (see Listing 13-8), and the result is written to the new .config file.
Compiling a New Kernel
Now that you have created the /usr/src/linux/.config file, it’s time to compile the new ker- nel, or better, build the new kernel. Whereas previously several commands were needed to do this, nowadays you can perform this task by running one simple command from the /usr/src/ linux directory:
make
Completing the build of a new kernel can take awhile. make will show you the names of all the individual source files that it is currently is working on and display any warning or status errors related to these files. Listing 13-9 gives you an impression of what you see at this point.
Although you won’t often have to recompile the kernel very often, the same is far from true for kernel modules. The issue is that many hardware vendors refuse to publish the source code for their drivers under open source licenses. Instead, they will make some proprietary Linux drivers available only. These proprietary Linux drivers are generic, but to be fully functional, drivers have to be developed for your specific kernel. The only way to meet this requirement is to compile these drivers for your machine.
Speaking in a generic way, there are two methods to install such drivers on your com- puter: the easy way and the hard way. When choosing the easy way, you’ll have to integrate the web site where the vendor made his or her drivers available as a package repository on your computer. When choosing the hard way, you’ll have to download and compile the drivers yourself. The exact procedure to do this is different for each driver. What is described in the following procedure is a generic way that will work in most cases:
- Download the driver and store it somewhere on your computer. Let’s assume that the driver you want to install is driver.tgz, and you have stored it in the home directory of the user root. You should also make sure that you have acquired root permissions before starting.
- Extract the tar archive using tar zxvf driver.tgz. Normally, this creates a sub- directory in the current directory, which would be /tmp/driver in this case. Use cd to change to this directory.
- In the driver subdirectory, you’ll normally find some files. One of them is the configure file (which may also have the name install, setup, or something similar). Often you’ll also find a file with the name Makefile, and in most cases there’s also a file with the name README. Be aware that exact file names may vary, so check to be sure that you are using the right files. It’s a good idea to start by reading the README file, as it may contain useful tips on how to install the driver.
- After reading the README file, run the generic setup script, which is normally configure. To run it from the current directory, run it as ./configure, and not just configure.
- The configure script makes sure that all conditions have been met to start compiling the driver.
- From the directory where you’ve extracted the driver files, run make to start the com- piling process. The make command will follow the instruction in the Makefile in the directory and compile the driver for your current kernel.
- After running make, you’ll need to make sure that all drivers are copied to the correct location. To do this, run make install, still within the directory that contains the driver files as the current directory.
This generic procedure should help you in compiling and installing the driver for your kernel. It is a very generic procedure, however: in all cases, you should check the documenta- tion that comes with the driver to see whether it contains any specific instructions.
When your computer starts up, a Linux kernel has to be loaded. On most Linux distributions, this is accomplished by the Grub 2 boot loader. This boot loader shows a configuration while it starts, and from this startup menu it allows you to specify specific options that should be used when booting (see Figure 13-4). It also has a configuration file that can be used to make startup parameters persistent.
Figure 13-4. The Grub2 bootloader menu
The Grub 2 boot menu shows the different kernel versions that are available. After software updates have been installed, you normally will see a few kernel version, which allows you to select an older version, or a troubleshooting kernel in case you have problems starting the default kernel. From the Grub 2 boot loader menu, you can press the e key on a highlighted item. This opens the screen that you can see in Figure 13-5, where kernel boot options can be specified. To do this, find the line that starts with the text Linux and add the boot parameters to the end of the line.
Figure 13-5. Specifying kernel boot options
In the example from figure 13-4 you can see that many arguments are provided on the line that starts with linux16. You’ll also notice that different Linux distributions are using different options. In this case for instance the options rhgb and quiet are used to indicate that the system should not show any startup messages. If you rather do want to see startup messages, you can remove these options from this line.
Also, you can add specific kernel options to this line. If your machine fails to start for instance, you can add the argument systemd.unit=rescue.target to enter rescue mode. This brings you in a minimal mode where you can repair settings on your system. Notice that all changes you’re making from here are once only. To start with the modified kernel arguments, use the Ctrl-X key stroke.
If you want to make changes to the GRUB 2 boot loader permanent, you should include them in the file /etc/default/grub. In this file you’ll find a line GRUB_CMDLINE that contains all kernel arguments that are used when booting. You can add to or remove from this line as required. After making modifications, you need to write the changes to the Grub 2 configuration. You can do so using grub2-mkconfig -o /boot/grub2/grub.cfg. Notice that you should never write to the /boot/grub2/grub.cfg file directly, as all modifications to it will be overwritten if the kernel or the Grub 2 software are updated.
EXERCISE 13-3: MODIFYING GRUB2 ARGUMENTS
In this exercise you’ll learn how to enter rescue mode.
- Use echo b > /proc/sysrq-trigger to reset your system. By writing a b to this file, you’re giving the system a hard reset.
- When the Grub 2 boot menu appears, select the default kernel entry and press e to open the edit mode.
- Find the line that starts linux16 and move your cursor to the end of this line. Add the text systemd.unit=rescue.target.
- Use the Ctrl-x key stroke to boot your system with these options. After logging in as root, you’ll be in a minimal troubleshooting mode.
- To enter a normal operational mode from troubleshooting mode, type systemctl isolate multi-user.target. This starts the normal multi-user mode on your system. Alternatively, you can type reboot to reboot your system.
Summary
In this chapter, you have learned how to work with the kernel. You’ve read how to manage ker- nel modules, as well as how to change kernel parameters. The following commands have been covered in this chapter:
- modprobe: Allows you to load and unload kernel modules
- modinfo: Provides information about kernel modules
- lsmod: Lists loaded modules
- insmod: Allows you to load kernel modules (legacy command)
- rmmod: Removes currently loaded modules from memory (legacy command)
- depmod: Creates the modules.dep configuration file, which makes sure that module dependencies are loaded automatically
- gcc: Refers to the GNU C compiler, used to convert source files into program files
- make: Works with the Makefile to make compiling software easier
- sysctl: Allows you to load kernel parameters while booting
- grub2-mkconfig: Writes changes from the /etc/default/grub file to the Grub 2 boot loader configuration.
In the next and final chapter of this book, you will read how to create shell scripts.