Chapter 22
Kernel and Module Management
A kernel is a complex piece of software that manages the processes and process interactions that take place within an operating system. As a user, you rarely, if ever, interact directly with it. Instead, you work with the applications that the kernel manages.
The Linux kernel is Linux. It is the result of years of cooperative (and sometimes contentious) work by numerous people around the world. There is only one common kernel source tree, but each major Linux distribution massages and patches its version slightly to add features, performance, or options. Each Linux distribution, including Ubuntu, comes with its own precompiled kernel as well as the kernel source code, providing you with absolute authority over the Linux operating system. This chapter covers the kernel and what it does for you and for the operating system.
In this chapter, you also learn how to obtain the kernel sources and how and when to patch the kernel. This chapter leads you through an expert’s tour of the kernel architecture and teaches you essential steps in kernel configuration, how to build and install modules, and how to compile drivers in Ubuntu. This chapter also teaches you important aspects of working with GRUB2, the default Ubuntu boot loader. Finally, the chapter’s troubleshooting information will help you understand what to do when something goes wrong with your Linux kernel installation or compilation process. As disconcerting as these problems can seem, this chapter shows you some easy fixes for many kernel problems.
Almost all users find that a precompiled Ubuntu kernel suits their needs (and there are several to choose from). At some point, you might need to recompile the kernel to support a specific piece of hardware or add a new feature to the operating system, although the Ubuntu kernel team works very hard to backport or enable any feature possible (as a module), so it is highly unlikely you will ever have a need to do this. The kernel team is also approachable and will gladly discuss specific needs and features. Sometimes features are not enabled just because no one has ever asked for or needed them. Occasionally, things are not enabled because of a conflict with another feature. The Ubuntu kernel team can help you discover what is going on in those cases (but don’t abuse their kindness and availability; they already work quite hard and for long hours).
Really, the main reason today that people compile their own kernel is because they want to learn to be a kernel developer. If you have heard horror stories about the difficulties of recompiling the Linux kernel, you can relax; this chapter gives you all the information you need to understand how to painlessly work through the process if you are interested in learning a new skill. This is a complex and detail-oriented task, but it is within the grasp of most technical users, even if it is completely unnecessary.
Caution
Building and using a custom kernel will make it difficult to get support for your system. Although it is a learning experience to compile your own kernel, you will not be allowed to file bugs in Ubuntu on a custom-built kernel (if you do, they will be rejected without further explanation), and if you have a commercial support contract with Ubuntu/Canonical, building and using a custom kernel will void the contract.
The Linux Kernel
The Linux kernel is the management part of the operating system that many people call Linux. Although many think of the entire distribution as Linux, the only piece that can correctly be called Linux is the kernel. Ubuntu, like many other Linux distributions, includes a kernel packaged with add-on software that interacts with the kernel so that the user can interface with the system in a meaningful manner.
The system utilities and user programs enable computers to become valuable tools to a user.
The First Linux Kernel
In 1991, Linus Torvalds released version 0.99 of the Linux kernel as the result of his desire for a powerful, UNIX-like operating system for his Intel 80386 personal computer. Linus wrote the initial code necessary to create what is now known as the Linux kernel and combined it with Richard Stallman’s GNU tools. Indeed, because many of the Linux basic system tools come from the GNU Project, many people refer to the operating system as GNU/Linux. Since then, Linux has benefited from thousands of contributors adding their talents and time to the Linux project. Linus still maintains the kernel, deciding what will and will not make it into the kernel as official releases, known to many as the vanilla or Linus Linux kernel.
The Linux Source Tree
The source code for the Linux kernel is kept in a group of directories called the kernel source tree. The structure of the kernel source tree is important because the process of compiling (building) the kernel is automated; it is controlled by scripts interpreted by the make application. These scripts, known as makefiles, expect to find the pieces of the kernel code in specific places; if they don’t find them, they will not work. You learn how to use make to compile a kernel later in this chapter.
It is not necessary for the Linux kernel source code to be installed on your system for the system to run or for you to accomplish typical tasks such as working with email, browsing the Web, or using a word processing program. It is necessary that the kernel sources be installed, however, if you want to compile a new kernel. In the next section, you learn how to install the kernel source files and how to set up the special symbolic link required. That link, /usr/src/linux-5.4.0, where the numbers at the end match the release version of the kernel installed on your system, is what we use to refer to the directory of the kernel source tree as we examine the contents of the kernel source tree.
Note
The pace of change in the Linux kernel has accelerated, much like the rest of our lives. In this chapter, we chose to use the version numbers for a recent LTS release of Ubuntu. The version numbers on your system may be different, but the processes and concepts remain the same.
The /usr/src/linux-5.4.0 directory contains the .config files and the makefiles, among others. The .config file is the configuration of your Linux kernel as it was compiled. There is no .config file by default; you must select one from the /configs subdirectory. There, you will find configuration files for each flavor of the kernel Ubuntu provides; simply copy the one appropriate for your system to the default directory and rename it .config.
We have already discussed the contents of the /configs subdirectory, so let’s examine the other directories found under /usr/src/linux-5.4.0. The most useful one for us is the Documentation directory. In it and its subdirectories, you will find almost all the documentation concerning every part of the kernel. The file 00-INDEX (which each Documentation subdirectory contains) contains a list of the files in the main directory and a brief explanation of what they are. Many files are written solely for kernel programmers and application writers, but a few are useful to intermediate or advanced Linux users when attempting to learn about kernel and device driver issues. Some of the more interesting and useful documents are as follows:
▸ devices.txt—This file contains a list of all possible Linux devices that are represented in the /dev directory, giving major and minor numbers and short descriptions. You may at some point have gotten an error message that mentions char-major-xxx; this file is where that list is kept.
▸ ide.txt—If your system uses IDE hard drives, this file discusses how the kernel interacts with them and lists the various kernel commands that you can use to solve IDE-related hardware problems, manually set data transfer modes, and otherwise manually manage your IDE drives. Most of this management is automatic, but if you want to understand how the kernel interacts with IDE devices, this file explains it.
▸ initrd.txt—This file provides much more in-depth knowledge of initial RAM disks, giving details on the loopback file system used to create and mount them and explaining how the kernel interacts with them.
▸ kernel-parameters.txt—This file is a list of most of the arguments that you can pass at boot time to configure kernel or hardware settings, but it does not appear too useful at first glance because it is just a list. However, knowing that a parameter exists and might relate to something you are looking for can assist you in tracking down more information because it gives you terms to enter into an Internet search engine such as Google.
▸ sysrq.txt—If you have ever wondered what that key on your keyboard marked SysRq is used for, this file has the answer. Briefly, it is a key combination hardwired into the kernel that can help you recover from a system lockup. Ubuntu disables this function by default for security reasons. You can re-enable it at a root prompt by entering the command # echo "1" > /proc/sys/kernel/sysrq, and you can disable it by echoing the value 0 rather than 1.
In the other directories under Documentation, you find similar text files that deal with the kernel modules for CD-ROM drivers, file system drivers, game port and joystick drivers, video drivers (not graphics card drivers; those belong to X11R6 and not to the kernel), network drivers, and all the other drivers and systems found in the Linux operating system. Again, these documents are usually written for programmers, but they can also provide useful information to intermediate and advanced Linux users.
The directory named scripts contains many of the scripts that make uses. It really does not contain anything of interest to anyone who is not a programmer or a kernel developer (also known as a kernel hacker).
After a kernel is built, all the compiled files wind up in the arch directory and its subdirectories. Although you can manually move them to their final location, you learn later in this chapter how the make scripts will do it for you. In the early days of Linux, this post-compilation file relocation was all done by hand; you should be grateful for make.
Note
The make utility is a complex program. You can find complete documentation on the structure of make files, as well as the arguments that it can accept, at www.gnu.org/software/make/manual/make.html.
The remaining directories in /usr/src/linux-5.4.0 contain the source code for the kernel and the kernel drivers. When you install the kernel sources, these files are placed there automatically. When you patch kernel sources, these files are altered automatically. When you compile the kernel, these files are accessed automatically. Although you never need to touch the source code files, they can be useful. The kernel source files are nothing more than text files with special formatting, which means you can look at them and read the programmer comments. Sometimes, a programmer writes an application but cannot (or often does not) write the documentation. The comments the programmer puts in the source code are often the only documentation that exists for the code.
Small testing programs are even “hidden” in the comments of some of the code, along with comments and references to other information. Because the source code is written in a language that can be read as easily—almost—as English, a nonprogrammer might be able to get an idea of what the application or driver is actually doing (see Chapter 38, “Using Programming Tools,” for an idea of how that could happen). This information might be of use to an intermediate to an advanced Linux user who is confronted by kernel- and driver-related problems.
Note
The interaction and control of hardware is handled by a small piece of the kernel called a device driver. The driver tells the computer how to interact with a modem, a SCSI card, a keyboard, a mouse, and so on in response to a user prompt. Without the device driver, the kernel does not know how to interact with the associated device.
Types of Kernels
In the early days of Linux, a kernel was a single block of code containing all the instructions for the processor, the motherboard, and the other hardware. If you changed hardware, you were required to recompile the kernel code to include what you needed and discard what you did not need. Including extra, unneeded code carried a penalty because the kernel became larger and occupied more memory. On older systems that had only 4MB to 8MB of memory, wasting precious memory for unnecessary code was considered unacceptable. Kernel compiling was something of a “black art,” as early Linux users attempted to wring the most performance possible from their computers. Such kernels compiled as a single block of code are called monolithic kernels.
As the kernel code grew larger and the number of devices that could be added to a computer increased, the requirement to recompile became onerous. A new method of building the kernel was developed to make the task of compiling easier. The part of the kernel’s source code that composed the code for the device drivers could be optionally compiled as a module that could be loaded and unloaded into the kernel as required. This is known as the modular approach to building the kernel. Now, all the kernel code could be compiled at once, with most of the code compiled into these modules. Only the required modules would be loaded; the kernel could be kept smaller, and adding hardware was much simpler.
The typical Ubuntu kernel has some drivers compiled as part of the kernel itself (called inline drivers) and others compiled as modules. Only device drivers compiled inline are available to the kernel during the boot process; modular drivers are available only after the system has been booted.
Note
As a common example, drivers for SCSI disk drives must be available to the kernel if you intend to boot from SCSI disks. If the kernel is not compiled with those drivers inline, the system does not boot because it cannot access the disks.
A way around this problem for modular kernels is to use an initial RAM disk (initrd), discussed later in this chapter, in the “Creating an Initial RAM Disk Image” section. initrd loads a small kernel and the appropriate device driver, which then can access the device to load the kernel you want to run.
Some code can be only one or the other (for technical reasons unimportant to the average user), but most code can be compiled either as modular or inline. Depending on the application, some system administrators prefer one way over the other, but with fast modern processors and abundant system memory, the performance differences are of little concern to all but the most ardent Linux hackers.
When compiling a kernel, making the selection of modular or inline is part of the make config step, detailed later in this chapter. Unless you have a specific reason to do otherwise, select the modular option when given a choice. Because you will be managing kernels more frequently than compiling kernels, the process of managing modules is addressed in the next section.
Managing Modules
When using a modular kernel, special tools are required to manage the modules. Modules must be loaded and unloaded, and it would be nice if that were done as automatically as possible. You also need to be able to pass necessary parameters to modules when you load them—things such as memory addresses and interrupts. (That information varies from module to module, so you need to look at the documentation for your modules to determine what, if any, information needs to be passed to them.) This section covers the tools provided to manage modules and then looks at a few examples of using them.
Linux provides the following module management tools, all of which (along with modprobe.conf) have man pages:
▸ lsmod—This command lists the loaded modules. It is useful to pipe this through the less command because the listing is usually more than one page long.
▸ insmod—This command loads the specified module into the running kernel. If a module name is given without a full path, the default location for the running kernel, /lib/modules/*/, is searched. Several options are offered for this command; the most useful is -f, which forces the module to be loaded.
▸ rmmod—This command unloads (removes) the specified module from the running kernel. More than one module at a time can be specified.
▸ modprobe—A more sophisticated version of insmod and rmmod, this command uses the dependency file created by depmod and automatically handles loading or, with the -r option, removing modules. There is no force option, however. A useful option to modprobe is -t, which causes modprobe to cycle through a set of drivers until it finds one that matches your system. If you are unsure of what module will work for your network card, use this command:
matthew@seymour:~$ sudo modprobe -t net
The term net is used here because that is the name of the directory (/lib/modules/*/kernel/net) where all the network drivers are kept. The modprobe command tries each one in turn until it loads one successfully.
▸ modinfo—This command queries a module’s object file and provides a list of the module name, author, license, and any other information that is there. It often is not very useful.
▸ depmod—This program creates a dependency file for kernel modules. Some modules need to have other modules loaded first; that is, they “depend” on the other modules. (A lot of the kernel code is like this because it eliminates redundancy in the code base.) During the boot process, one of the startup files contains the command depmod -a, and it is run every time you boot to re-create the file /lib/modules/*/modules.dep. If you make changes to the file /etc/modprobe.conf, run depmod -a manually. The depmod command, its list of dependencies, and the /etc/modprobe.conf file enable kernel modules to be automatically loaded as needed.
▸ /etc/modprobed—This directory contains a set of *.conf files to specify options for modules that have dependencies. They can also be used to create aliases or override the default modprobe behavior for modules with special requirements.
▸ /etc/modprobe.conf—This is not a command but a file that controls how modprobe and depmod behave; it contains kernel module variables. Although the command syntax can be quite complex, most actual needs are simple. The most common use is to alias a module and then pass some parameters to it. For example, the following code aliases a device name (from devices.txt) to a more descriptive word and then passes some information to an associated module:
alias char-major-89 i2c-dev options eeprom ignore=2,0x50,2,0x51,2,0x52
The i2c-dev device is used to read the CPU temperature and fan speed on this system. These lines for /etc/modprobe.conf were suggested for our use by the program’s documentation. We added them with a text editor.
A partial listing of lsmod is shown here, piped through the less command, enabling you to view it a page at a time:
matthew@seymour:~$ sudo lsmod | less Module Size Used by parport_pc 19392 1 Module Size Used by parport_pc 19392 1 lp 8236 0 joydev 17377 0 parport 29640 2 parport_pc,lp autofs4 10624 0 sunrpc 101064 1
The list is actually much longer, but here you see that the input module is being used by the joydev (joystick device) module, but the joystick module is not being used. This computer has a joystick port that was autodetected, but no joystick is connected. A scanner module is also loaded, but because the USB scanner is unplugged, the module is not being used. You use the lsmod command to determine whether a module was loaded and what other modules were using it. If you examine the full list, you see modules for all the devices attached to your computer.
To remove a module, joydev in this example, use the following:
matthew@seymour:~$ sudo rmmod joydev
or use this:
matthew@seymour:~$ sudo modprobe -r joydev
The output of lsmod now shows that it is no longer loaded. If you were to remove input as well, you could then use modprobe to load both input and joydev (one depends on the other, remember) with a simple command, as follows:
matthew@seymour:~$ sudo modprobe joydev
If Ubuntu balks at loading a module (because it was compiled using a different kernel version from what you are currently running; for example, the NVIDIA graphics card module), you could force it to load like this:
matthew@seymour:~$ sudo insmod -f nvidia
You ignore the complaints (error messages) in this case if the kernel generates any.
When to Recompile
Ubuntu systems use a modified version of the plain-vanilla Linux kernel (a modified version is referred to as a patched kernel) with additional drivers and other special features compiled into it.
Ubuntu has quite an intensive testing period for all distribution kernels and regularly distributes updated versions. The supplied Ubuntu kernel is compiled with as many modules as possible to provide as much flexibility as possible. A running kernel can be further tuned with the sysctl program, which enables direct access to a running kernel and permits some kernel parameters to be changed. As a result of this extensive testing, configurability, and modularity, the precompiled Ubuntu kernel does everything most users need it to do. Most users only need to recompile the kernel to do the following:
▸ Accommodate an esoteric piece of new hardware
▸ Conduct a system update when Ubuntu has not yet provided precompiled kernels
▸ Experiment with the system capabilities
Ubuntu supplies precompiled versions of the kernel for 32- and 64-bit processors. For each architecture, Ubuntu compiles a generic kernel that works well for most uses, a server kernel that is optimized for server use, a preempt kernel designed for use in low-latency servers, and an rt kernel for times when instant response is more important than balanced use (such as in professional audiovisual recording and editing). There is also a special kernel, called virtual, available for use in virtual machines. These are all available from the Ubuntu software repositories.
Also available are a series of packages called linux-backports-modules-, each with a specific set of kernel modules backported from newer mainline kernels into current version Ubuntu kernels. If you need an updated driver for a piece of hardware, look at the backported modules first.
Kernel Versions
The Linux kernel is in a constant state of development. As new features are added, bugs are fixed, and new technology is incorporated into the code base, it becomes necessary to provide stable releases of the kernel for use in a production environment. Having separate releases that contain the newest code for developers to test is also important. To keep track of the kernels, version numbers are assigned to them. Programmers enjoy using sequential version numbers that have abstract meaning. Is version 8 twice as advanced as version 4 of the same application? Is version 1 of one application less developed than version 3 of another? The version numbers cannot be used for this kind of qualitative or quantitative comparison. It is entirely possible for higher version numbers to have fewer features and more bugs than older versions. The numbers exist solely to differentiate and organize sequential revisions of software. The kernel version can be broken down into four sections:
▸ Major version—This is the major version number, which is 5 in our examples.
▸ Minor version—This is the minor version number, which is 4 in our examples.
▸ Sublevel number—This number indicates the current iteration of the kernel; which is 0 in our examples.
▸ Extraversion level—This is the number representing a collection of patches and additions made to the kernel by the Ubuntu engineers to make the kernel work for them (and you). Each collection is numbered, and the number is indicated here in the kernel name. There is not one in our preceding example. In the following example, it is 9.
Type uname -r at the command prompt to display your current kernel version, shown here with sample output:
matthew@seymour:~$ uname -r 5.4.0.9-generic
Obtaining the Kernel Sources
The Linux kernel has always been freely available to anyone who wants it. If you just want to recompile the existing kernel, install the linux-source package from the Ubuntu repositories. To get the very latest vanilla version (which is the commonly used term for the kernel version direct from the main kernel developers and which has not yet been patched or changed by any distribution-specific kernel team), open an HTTP connection to www.kernel.org/pub/ or use Git to retrieve from https://git.kernel.org or use RSYNC to retrieve from rsync://rsync.kernel.org/pub/. We will use kernel 5.4.0 in the following examples.
A number of different entries exist on the archive site for each kernel version, but because you are interested only in the full kernel, getting the full package of source code is necessary (for example, linux-5.4.0.bz2).
The .bz2 extension is applied by the bzip2 utility, which has better compression than gzip.
After it is downloaded, move the package to a directory other than /usr/src and unpack it. The bzip2 unpack command is tar -xjvf linux-5.4.0.tar.bz2. After it is unpacked, the package creates a new directory, linux-5.4.0. Copy it to /usr/src or move it there. Then create a symbolic link of linux-4.15 to linux-5.4.0. (Otherwise, some scripts will not work.) Here is how to create the symbolic link:
matthew@seymour:~$ sudo rm /usr/src/linux-5.4 matthew@seymour:~$ sudo ln -s /usr/src/linux-5.4.0 /usr/src/linux-5.4
By creating a symbolic link to /usr/src/linux-5.4, it is possible to allow multiple kernel versions to be compiled and tailored for different functions: You just change the symbolic link to the kernel directory you want to work on.
Caution
The correct symbolic link is critical to the operation of make. Always have the symbolic link point to the version of the kernel sources you are working with.
Patching the Kernel
Patching a kernel to the newest Linux kernel version as opposed to downloading the entire source code is possible. This choice can be beneficial for those who are not using a high-speed broadband connection. Whether you are patching existing sources or downloading the full source, the end results are identical.
Patching the kernel is not a mindless task. It requires the user to retrieve all patches from the current version to the version the user wants to upgrade to. For example, if you are currently running 5.3.3 (and have those sources) and want to upgrade to 5.4.0, you must retrieve the 5.3.3 all the way up to the 5.4.0 patch set, and so on. After you download these patches, you must apply them in succession to upgrade to 5.4.0. This is more tedious than downloading the entire source, but it’s useful for those who keep up with kernel hacking and want to perform incremental upgrades to keep their Linux kernel as up to date as possible.
To patch up to several versions in a single operation, you can use the patch-kernel script located in the kernel source directory for the kernel version you currently use. This script applies all necessary version patches to bring your kernel up to the latest version.
The format for using the patch-kernel script looks like this:
patch-kernel source_dir patch_dir stopversion
The source directory defaults to /usr/src/linux if none is given, and patch_dir defaults to the current working directory if one is not supplied.
For example, assume that you have a 5.3.3 kernel code tree that needs to be patched to the 5.4.0 version. The needed earlier and 5.4.0 patch files have been downloaded and placed in the /patch directory in the source tree. You issue the following command in the /usr/src/linux-5.4 directory:
matthew@seymour:~$ sudo scripts/patch-kernel /usr/src/linux-5.4 /usr/src/ linux-5.4.0/patch
Each successive patch file is applied, eventually creating a 5.4.0 code tree. If any errors occur during this operation, files named xxx# or xxx.rej are created, where xxx is the version of the patch that failed. You have to resolve these failed patches manually by examining the errors and looking at the source code and the patch. An inexperienced person will not have any success with this because you need to understand C programming and kernel programming to know what is broken and how to fix it. Because this was a stock 4.1.1 code tree, the patches were all successfully applied without errors. If you are attempting to apply a nonstandard third-party patch, the patch might fail.
When you have successfully patched the kernel, you are ready to begin compiling this code tree as if you were starting with a fresh, stock 5.4.0 kernel tree.
Using the Patch Command
If you have a special, nonstandard patch to apply—such as a third-party patch for a commercial product—you can use the patch command rather than the special patch-kernel script that is normally used for kernel source updates. Here are some quick steps and an alternative method of creating patched code and leaving the original code alone:
Create a directory in your home directory and name it something meaningful, like mylinux.
Copy the pristine Linux source code there with the following:
cp -ravd /usr/src/linux-4.15/* ~/mylinux
Copy the patch file to that same directory as follows:
cp patch_filename ~/mylinux
Change to the ~/mylinux directory with this command:
cd ~/mylinux
Apply the patch like this:
-patch -p1 < patch_filename > mypatch.log 2>&1
(This last bit of code saves the message output to a file so that you can look at it later.)
If the patch applies successfully, you are done and have not endangered any of the pristine source code. If the newly patched code does not work, you do not have to reinstall the original, pristine source code.
Copy your new code to /usr/src and make that special symbolic link described elsewhere in the chapter.
Compiling the Kernel
If you want to update the kernel from new source code you have downloaded, or if you have applied a patch to add new functionality or hardware support, you must compile and install a new kernel to actually use that new functionality. Compiling the kernel involves translating the kernel’s contents from human-readable code to binary form. Installing the kernel involves putting all the compiled files where they belong in /boot and /lib and making changes to the boot loader.
The process of compiling the kernel, like the process of installing it, is almost completely automated by the make utility. By providing the necessary arguments and following the steps covered next, you can recompile and install a custom kernel for your use:
Do not delete your current kernel, so that you will have a backup that you can use to boot if there is a problem with the one you compile.
Apply all patches, if any, so that you have the features you desire. See the previous section for details.
Back up the
.configfile, if it exists, so that you can recover from the inevitable mistake. Use the followingcpcommand to do so:matthew@seymour:~$ sudo cp .config .config.bak
Note
If you are recompiling the Ubuntu default kernel, the
/usr/src/linux-4.15/configsdirectory contains several versions of configuration files for different purposes.Ubuntu provides a full set of .config files in the subdirectory configs, all named for the type of system they were compiled for. If you want to use one of these default configurations as the basis for a custom kernel, just copy the appropriate file to /usr/src/linux-4.15 and rename it .config.
Run the
make mrproperdirective to prepare the kernel source tree, cleaning out any old files or binaries.Restore the
.configfile that the commandmake mrproperdeleted and edit the makefile to change theEXTRAVERSIONnumber.Note
If you want to keep any current version of the kernel that was compiled with the same code tree, manually edit the makefile with your favorite text editor and add some unique string to the
EXTRAVERSIONvariable.You can use any description you prefer.
Modify the kernel configuration file using
make config,make menuconfig, ormake xconfig; we recommend the last one.Run
make depto create the code dependencies used later in the compilation process.Tip
If you have a multiprocessor machine, you can use both processors to speed the
makeprocess by inserting-jxafter themakecommand, where, as a rule of thumb,xis one more than the number of processors you have. You might try a larger number and even try this on a single processor machine (we have used-j8successfully on an SMP machine); it will only load up your CPU. Here is an example:matthew@seymour:~$ sudo make –j3 bzImage
All the make processes except make dep work well with this method of parallel compiling.
Run
make cleanto prepare the sources for the compilation of the kernel.Run
make bzImageto create a binary image of the kernel.Note
Several choices of directives exist; the most common ones are the following:
▸
zImage—This directive compiles the kernel, creating an uncompressed file calledzImage.▸
bzImage—This directive creates a compressed kernel image necessary for some systems that require the kernel image to be under a certain size for the BIOS to be able to parse them; otherwise, the new kernel will not boot. It is the most commonly used choice.Run
make modulesto compile any modules your new kernel needs.Run
make modules_installto install the modules in/lib/modulesand create dependency files.Run
make installto automatically copy the kernel to/boot, create any other files it needs, and modify the boot loader to boot the new kernel by default.Using your favorite text editor, verify the changes made to
/etc/lilo.confor/boot/grub/grub.conf; fix if necessary and rerun/sbin/liloif needed.Reboot and test the new kernel.
Repeat the process if necessary, choosing a configuration interface.
Over time, the process for configuring the Linux kernel has changed. Originally, you configured the kernel by responding to a series of prompts for each configuration parameter; this is the make config utility described shortly. Although you can still configure Linux this way, most users find that type of configuration confusing and inconvenient; moving back through the prompts to correct errors, for instance, is impossible.
The make config utility is a command-line tool. The utility presents a question about kernel configuration options. The user responds with a Y, N, M, or ?. (It is not case sensitive.) Responding M configures the option to be compiled as a module. A response of ? displays context help for that specific options, if available. (If you choose ? and no help is available, you can turn to the vast Internet resources to find information.) We recommend that you avoid the make config utility.
If you prefer to use a command-line interface, you can use make menuconfig to configure the Linux kernel. menuconfig provides a graphical wrapper around a text interface. Although it is not as raw as make config, menuconfig is not a fancy graphical interface either; you cannot use a mouse but must navigate through it using keyboard commands. The same information presented in make config is presented by make menuconfig, but it looks a little nicer. Now, at least you can move back and forth in the selection process if you change your mind or make a mistake.
In make menuconfig, you use the arrow keys to move the selector up and down and the spacebar to toggle a selection. The Tab key moves the focus at the bottom of the screen to either Select, Exit, or Help.
If a graphical desktop is not available, menuconfig is the best you can do. However, both menuconfig and xconfig (see the following explanation of each) are improvements over editing the .config file directly. If you want to configure the kernel through a true graphical interface—with mouse support and clickable buttons—make xconfig is the best configuration utility option. To use this utility, you must have the X Window System running. The application xconfig is really nothing but a Tcl/Tk graphics widget set that provides borders, menus, dialog boxes, and the like. Its interface is used to wrap around data files that are parsed at execution time.
After loading this utility, you use it by clicking the buttons that list the configuration options. Each button you click opens another window that has the detailed configuration options for that subsection. Three buttons are at the bottom of each window: Main Menu, Next, and Prev(ious). Clicking the Main Menu button closes the current window and displays the main window. Clicking Next takes you to the next configuration section. When configuring a kernel from scratch, click the button labeled Code Maturity Level Options and then continue to click the Next button in each subsection window to proceed through all the kernel configuration choices. When you have selected all options, the main menu is again displayed. The buttons on the lower right of the main menu are for saving and loading configurations. Their functions are self-explanatory. If you just want to have a look, go exploring! Nothing will be changed if you elect not to save it.
If you are upgrading kernels from a previous release, going through the entire configuration from scratch is not necessary. Instead, you can use the directive make oldconfig; it uses the same text interface that make config uses, and it is noninteractive. It just prompts for changes for any new code.
Using xconfig to Configure the Kernel
For simplicity’s sake, during this brisk walkthrough, this discussion assumes that you are using make xconfig and that prior to this point, you have completed the first five steps in the kernel compilation list shown previously.
As you learned in the preceding section, you configure the kernel using make xconfig by making choices in several configuration subsection windows. Each subsection window contains specific kernel options. With hundreds of choices, configuring the kernel is daunting. We cannot really offer you detailed descriptions of which options to choose because our configuration will not match your own system and setup.
Table 22.1 provides brief descriptions of many of the subsections and options so that you can get an idea of what you might encounter. We recommend that you copy your kernel’s .config file to /usr/src/linux-4.15 and run make xconfig from there. Explore all the options. As long as you do not save the file, absolutely nothing is changed on your system.
Table 22.1 Some Kernel Subsections for Configuration
Subsection |
Description |
|---|---|
Code maturity level options |
Enables development code to be compiled into the kernel even if it has been marked as obsolete or as testing code only. This option should be used only by kernel developers or testers because of the possible unusable state of the code during development. |
General setup |
This section contains several different options covering how the kernel talks to the BIOS, whether it should support PCI or PCMCIA, whether it should use APM or ACPI, and what kind of Linux binary formats will be supported. Contains several options for supporting kernel structures necessary to run binaries compiled for other systems directly, without recompiling the program. |
Loadable module support |
Determines whether the kernel enables drivers and other nonessential code to be compiled as loadable modules that can be loaded and unloaded at runtime. This option keeps the basic kernel small so that it can run and respond more quickly; in that regard, choosing this option is generally a good idea. |
Processor type and features |
Several options dealing with the architecture that will be running the kernel. |
Power management options |
Options dealing with ACPI and APM power management features. |
Bus options |
Configuration options for the PCMCIA bus found in laptops and PCI hotplug devices. |
Memory technology devices |
Options for supporting flash memory devices, such as (MTD) EEPROMS. Generally, these devices are used in embedded systems. |
Parallel port support |
Several options for configuring how the kernel will support parallel port communications. |
Plug-and-play configuration |
Options for supporting plug-and-play (PnP) PCI, ISA, and PnP BIOS support. Generally, supporting PnP for PCI and ISA devices is a good idea. |
Block devices |
Section dealing with devices that communicate with the kernel in blocks of characters instead of streams. This includes IDE and ATAPI devices connected via parallel ports, as well as enabling network devices to communicate as block devices. |
ATA/IDE/MFM/RLL support |
Large collection of options to configure the kernel to communicate using different types of data communication protocols to talk to mass storage devices, such as hard drives. Note that this section does not cover SCSI. |
SCSI device support |
Options for configuring the kernel to support Small Computer System Interface (SCSI). This subsection covers drivers for specific cards, chipsets, and tunable parameters for the SCSI protocol. |
Old CD-ROM drivers |
Configuration options to support obscure, older CD-ROM devices that do not conform to the SCSI or IDE standards. These are typically older CD-ROM drivers that are usually a proprietary type of SCSI (not SCSI and not IDE). |
Multidevice support |
Options for enabling the kernel to support RAID devices in RAID and LVM software emulation and the different levels of RAID. Also contains options for support of a logical volume manager. |
Fusion MPT device support |
Configures support for LSI’s Logic Fusion Message Passing Technology for high-performance SCSI and LAN interfaces. |
IEEE1394 (firewire) support |
Experimental support for FireWire devices. |
I20 device support |
Options for supporting the Intelligent Input/Output architecture, which enables the hardware driver to be split from the operating system driver, thus enabling a multitude of hardware devices to be compatible with an operating system in one implementation. |
Networking support |
Several options for the configuration of networking in the kernel. The options are for the types of supported protocols and configurable options of those protocols. |
Amateur radio support |
Options for configuring support of devices that support the AX25 protocol. |
IrDA (infrared) support |
Options for configuring support of the infrared Data Association suite of protocols and devices that use these protocols. |
Bluetooth support |
Support for the Bluetooth wireless protocol. Includes options to support the Bluetooth protocols and hardware devices. |
ISDN subsystem |
Options to support Integrated Services Digital Network (ISDN) protocols and devices. ISDN is a method of connection to a large area network digitally over conditioned telephone lines, largely found to connect users to ISPs. |
Telephony support |
Support for devices that enable the use of regular telephone lines to support Voice over Internet Protocol (VoIP) applications. This section does not handle the configuration of modems. |
Input device support |
Options for configuring Universal Serial Bus (USB) human interface devices (HID) such as keyboards, mice, and joysticks. |
Character devices |
Configuration options for devices that communicate to the server in sequential characters. This is a large subsection containing the drivers for several motherboard chipsets. |
Multimedia devices |
Drivers for hardware implementations of video and sound devices such as video capture boards, TV cards, and AM/FM radio adapter cards. |
Graphics support |
Configuration options for VGA text console, video mode selection, and support for frame buffer cards. |
Sound |
Large subsection to configure supported sound card drivers and chipset support for the kernel. |
USB support |
USB configuration options. Includes configuration for USB devices and vendor-specific versions of USB. |
File system |
Configuration options for supported file system types. |
Additional device driver support |
A section for third-party patches. |
Profiling support |
Profiling kernel behavior to aid in debugging and development. |
Kernel hacking |
A section that determines whether the kernel will contain advanced debugging options. Most users will not want to include this option in their production kernels because it increases the kernel size and slows performance by adding extra routines. |
Security options |
Configuration options for whether the NSA’s Security Enhanced Linux (SELinux) is enabled. |
Cryptographic options |
Support for cryptography hardware. (Ubuntu patches not found in the vanilla kernel sources.) |
Library routines |
Options for |
After you select all the options you want, you can save the configuration file and continue with step 7 in the kernel compilation steps list shown earlier.
Creating an Initial RAM Disk Image
If you require special device drivers to be loaded to mount the root file system (for SCSI drives, network cards, or exotic file systems, for example), you must create an initial RAM disk image named /boot/initrd.img. For most users, creating this file is not necessary, but if you are not certain, having it really does not hurt. To create an initrd.img file, use the shell script /sbin/mkinitrd. The format for the command is the following, where file_name is the name of the image file you want to create:
/sbin/mkinitrd file_name kernel_version
mkinitrd looks at /etc/fstab, /etc/modprobe.conf, and /etc/ raidtab to obtain the information it needs to determine which modules should be loaded during boot. For our system, we use the following:
matthew@seymour:~$ sudo mkinitrd initrd-5.4.0.img 5.4.0-1
When Something Goes Wrong
Several things might go wrong during a kernel compile and install, and several clues point to the true problem. You may see error messages printed to the screen, and some error messages are printed to the file /var/log/syslog, which you can examine with a text editor. If you have followed the directions for patching the kernel, you need to examine a special error log as well. Do not worry about most errors because many problems are easily fixed with some research on your part. Some errors may be unfixable, however, depending on your skill level and the availability of technical information.
Errors During Compile
Although it is rare that the kernel will not compile, there is always a chance that something has slipped though the regression testing. Let’s take a look at an example of a problem that might crop up during the compile.
It is possible that the kernel compile may crash and not complete successfully, especially if you attempt to use experimental patches, add untested features, or build newer and perhaps unstable modules on an older system.
At this juncture, you have two options:
▸ Fix the errors and recompile.
▸ Remove the offending module or option and wait for the errors to be fixed by the kernel team.
Most users will be unable to fix some errors because of the complexity of the kernel code, although you should not rule out this option. It is possible that someone else discovered the same error during testing of the kernel and developed a patch for the problem, so you can check the Linux kernel mailing list archive. If the problem is not mentioned there, a search on Google might turn up something.
The second option, removing the code, is the easiest and is what most people do in cases in which the offending code is not required. In the case of the NTFS module failing, it is almost expected because NTFS support is still considered experimental and subject to errors. This is primarily because the code for the file system is reverse-engineered instead of implemented via documented standards. Read-only support has gotten better in recent kernels; write support is still experimental.
Finally, if you want to take on the task of trying to fix the problem yourself, this is a great opportunity to get involved with the Linux kernel and make a contribution that could help many others.
If you are knowledgeable about coding and kernel matters, you might want to look in the Maintainers file in the /usr/src/linux-5.4/ directory of the kernel source and find the maintainer of the code. The recommended course of action is to contact the maintainer to see if the maintainer is aware of the problems you are having. If nothing has been documented for the specific error, submitting the error to the kernel mailing list is an option. The guidelines for doing this are in the README file in the base directory of the kernel source under the section IF SOMETHING GOES WRONG.
Runtime Errors, Boot Loader Problems, and Kernel Oops
Runtime errors occur as the kernel is loading. Error messages are displayed on the screen or are written to the /var/log/syslog file. Boot loader problems display messages to the screen; no log file is produced. Kernel oops are errors in a running kernel, and error messages are written to the /var/log/syslog file.
Excellent documentation exists on the Internet for troubleshooting just about every type of error that GRUB2 or the kernel could give during boot. The best way to find this documentation is to go to your favorite search engine and type in the keywords of the error you received. Adjust the keywords you use as you focus your search.
If you have GRUB problems, see the GRUB manual online at www.gnu.org/software/grub/manual/ and look for further information at https://help.ubuntu.com/community/Grub2.
References
▸ www.kernel.org—The Linux Kernel Archives, the source of all development discussion for the Linux kernel
▸ www.gnu.org—A source for manuals and software for programs used throughout the kernel compilation process as well as official documentation for tools such as make and gcc
▸ https://wiki.ubuntu.com/Kernel—The starting point for anything you want to know about both Ubuntu and its use of the Linux kernel
▸ https://wiki.ubuntu.com/Kernel/FAQ—The Ubuntu kernel team’s answers to the most commonly asked questions about Ubuntu’s use of Linux kernels
▸ www.tldp.org—The Linux Documentation Project, the mecca of all Linux documentation and an excellent source of HOWTO documentation as well as FAQs and online books about Linux
▸ www.minix3.org—The MINIX website, which contains a selection of links to information about MINIX, a free, open-source, operating system designed to be highly reliable, flexible, and secure