You can find the current releases of the kernel at http://kernel.org (see Figure 11-1). This site is the canonical location of the Linux kernel sources and the hub of activity for the kernel project. Present on the site is every Linux distribution released along with links to the various mailing lists you can use to follow and participate in the project.

There are three ways for you to get the kernel sources. Each is described in a brief section following the summary:

Figure 11.1. The current releases of the kernel can be found at http://kernel.org

tar xzf linux-2.6.30.1.tar.gz

$ git clone git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-2.6.31.y.git

The kernel configuration process works when the kernel configuration program reads a kernel configuration specification file, called the kconfig file. This file resides in the arch directory, as specified with the ARCH=<your architecture> parameter when you run menuconfig. The kconfig file for the architecture contains the settings specific to the architecture and then includes configuration files for the rest of the kernel. For example, the MIPS kconfig file contains the following:

config MIPS
        bool
        default y
        select HAVE_IDE
        select HAVE_OPROFILE
        select HAVE_ARCH_KGDB
        # Horrible source of confusion. Die, die, die ...
        select EMBEDDED
        select RTC_LIB

mainmenu "Linux/MIPS Kernel Configuration"

menu "Machine selection"

(many, many lines clipped)

source "net/Kconfig"
source "drivers/Kconfig"

source "fs/Kconfig"
source "arch/mips/Kconfig.debug"
source "security/Kconfig"
source "crypto/Kconfig"
source "lib/Kconfig"

The kernel configuration program reads in the .config file for the kernel, applies the settings to the kconfig that was just read into the system, and renders the menu. After you select options, the changes are written back out to the .config file. Early in the build process, the file include/linux/autoconf.h is generated: it contains all the settings in the .config file as C preprocessor macros. The autoconf.h file is included in all the files in the Linux kernel; that is how the options you select in the configuration program are made available to the kernel. To see the process in terms of actions and configuration files, have a look at Figure 11-3.

Figure 11.3. How the Linux kernel configuration process works

The settings for the current configuration session are kept in the .config file in the top level of the kernel sources. Default configuration files (called defconfig files) for boards are stored under their respective arch directories. To get a listing of the defconfig files for a given architecture, do the following from the top-level kernel directory:

$ make ARCH=<your architecture> help

Some generic help for building the kernel appears, followed by a listing of the defconfig files for <your architecture>. In this case, the value was mips:

pnx8335-stb225_defconfig - Build for pnx8335-stb225
  pnx8550-jbs_defconfig    - Build for pnx8550-jbs
  pnx8550-stb810_defconfig - Build for pnx8550-stb810
  rb532_defconfig          - Build for rb532

These defconfig files are nothing more than .config files that has been copied to a different directory and renamed. The kernel build system expects these files to be in the configs directory under the selected architecture; when the make target matches a file in the defconfig directory, it copies the file to .config.

If you're not interested in running make to see the default configurations available, or if you want more information than make provides (such as a date, or if you'd like to sort the list), you can locate these files with the find utility:

$ find arch/<your architecture> -name "*_defconfig"

Using mips for the architecture, the following appears:

(clipped)
arch/mips/configs/msp71xx_defconfig
arch/mips/configs/ip32_defconfig
arch/mips/configs/ip27_defconfig
arch/mips/configs/jazz_defconfig
arch/mips/configs/yosemite_defconfig
(clipped)

To use one of these files to configure the kernel, use it as a target for make. For example, to configure the kernel to build for the yosemite board, you issue the following command from the top level of the kernel directory:

$ make ARCH=mips yosemite_defconfig
#
# configuration written to .config
#

The kernel will be ready to compile, targeting this processor. Because a defconfig file is just a renamed and relocated .config file, you can also store the .config file used in your project in the configs directory.

Because the .config file is a plain-text file with an uncomplicated format, editing it directly is a temptation and is sometimes the fastest way to change a value that's buried in the configuration's menu system. There is nothing wrong with editing this file by hand, and doing so is common among developers who know exactly what change they want to make. As an example, developers frequently change the value of CONFIG_CMDLINE, the kernel command line; opening the file in an editor and changing the line is much faster than using the existing kernel configuration tools.

After you make a change to the .config file, the best practice is to use the oldconfig target to test the selected options against the current configuration settings. For example,

$ make ARCH=arm oldconfig

results in the current .config file being processed. Any problems are reported on the command line. For example, some changes to the file result in other dependent options no longer being valid.

Another practical use for oldconfig is when you use the contents of a kernel's /proc/config.gz file. Recall from Chapter 4 that when examining a board, you can get a copy of the configuration used to build the kernel from the kernel itself. If you want to build the kernel with that file, the steps are e as follows:

After the kernel is configured, it's ready to be built. The output, or target, of a kernel build is a binary suitable for the boot loader on the target board, because the boot loader is the program responsible for getting the kernel code in memory and ready to be executed. The build also produces some of the tools used to build the kernel itself; unlike other projects, the build does the right thing when compiling programs for the host (such as those supporting programs) and object code for the target, such as the kernel and modules.

Many different targets are available for building a kernel, and the right one depends on the boot loader for the board. The differences between the targets in terms of the overall Linux build is quite small, because the build targets determine what happens in the last few steps of the build to format the object code so that it can be used with the boot loader on the board. Table 11-3 outlines the targets available and their applicability.

In addition to picking the appropriate target, you also need to tell the make file what compiler to use. The kernel's make file cross-compiles well, and the make file uses the host system's compiler when building tools for building the kernel and the cross-compiler when building code. The general case command line looks something like this:

$ make ARCH=<arch> CROSS_COMPILER=<cross compiler prefix> <target>

To build a kernel for a PowerPC board using a U-Boot boot loader, use the following command:

$ make uImage ARCH=powerpc CROSS_COMPILER=powerpc-405-linux-gnuabi-

This command line specifies the cross-compiler; it begins with powerpc-405-linux-gnuabi-. The value for CROSS_COMPILER can include a fully qualified directory path if you don't want to put the cross-compiler on the path. Some users who build several different kernels regularly opt for the fully qualified name route; however, putting the cross-compiler on the path is just as effective.

If you skipped ahead to this section and didn't configure the kernel, the following message appears:

***
*** You have not yet configured your kernel!
*** (missing kernel .config file)
***
*** Please run some configurator (e.g. "make oldconfig" or
*** "make menuconfig" or "make xconfig").
***

After the compilation process finishes, the following appears, to show what was produced (this output will vary slight depending on your kernel):

SYMLINK include/asm -> include/asm-powerpc
  CALL    scripts/checksyscalls.sh
  CHK     include/linux/compile.h
  CALL    arch/powerpc/kernel/systbl_chk.sh
  CC      arch/powerpc/kernel/prom_init.o
  CALL    arch/powerpc/kernel/prom_init_check.sh
  WRAP    arch/powerpc/boot/uImage
Image Name:   Linux-2.6.29.4
Created:      Wed Jun 10 23:19:58 2009
Image Type:   PowerPC Linux Kernel Image (gzip compressed)
Data Size:    1467720 Bytes = 1433.32 kB = 1.40 MB
Load Address: 0x00000000
Entry Point:  0x00000000

The output of the kernel build is located under the arch/<ARCH>/boot directory. The actual location varies, so double-check what the output of the build says. In this example, the line

WRAP    arch/powerpc/boot/uImage

shows that the output of the build is arch/powerpc/boot/uImage. If you're new to the kernel, this process is a bit mysterious, because no message says "Your Kernel is Here!" The rule of thumb is to look for the line that has a file name the same as the target under the arch/<ARCH>/boot directory. In this case, the uImage file produced by the build is ready for download to a target board.

A kernel module is code that is linked into the kernel while the kernel is running. When you're building the kernel, you can either link drivers directly into the kernel or build them as modules that can be loaded into the kernel at some other time. Most embedded systems don't use kernel modules because the hardware on the board is fixed and doesn't change over the life of the system. That is, the kernel that is built for an embedded system is for that system and no other. Compare this to your desktop system, where the kernel is intended to run on an arbitrary x86 host and must adapt by loading modules. Your desktop system probably has dozens of kernel modules loaded. On your host machine, do the following to see a list of the installed modules:

$ lsmod
Module                        Size  Used by
usbhid                     42336  0
nls_iso8859_1              12032  0
nls_cp437                  13696  0
vfat                       18816  0
fat                        58272  1 vfat
mmc_block                  17668  0
aes_i586                   15744  0
aes_generic                35880  1 aes_i586
ftdi_sio                   56968  0
i915                       65540  2

Remember, a general-purpose Linux distribution doesn't know what devices will be on the machine, so the kernel has a minimal set of drivers statically linked and then uses modules, which are dynamically linked, to get the remaining drivers. This minimal set of drivers is enough to drive a display and mount a few file systems likely to be available on most hardware platforms.

The work involved in looking at the current hardware configuration and loading the drivers also takes time during the boot process. Many embedded devices don't have much time before they're up and running, so the time overhead alone may disqualify the use of kernel modules.

For devices that do use kernel modules, building the modules is a straightforward task. The prior section showed how to build a kernel for a PowerPC 405-based board; using that same kernel directory, you build the modules with the following command:

$ make modules ARCH=powerpc CROSS_COMPILE=powerpc-405-linux-gnu-
  (clipped)
  CC [M]  drivers/of/of_i2c.o
  CC [M]  drivers/video/fbmem.o
  CC [M]  drivers/video/fbmon.o
  CC [M]  drivers/video/fbcmap.o
  (clipped)

The output shows that the process is busy building modules (by way of the [M]). The output of a kernel module build is some number of files ending in .ko that are scattered across the kernel's source tree.

After they're compiled, the modules can be gathered into one location via an installation target in the kernel's make file. Installing the modules for an embedded system means gathering the .ko files in the kernel source tree and putting them in a directory. That directory's contents are then moved to the target device. You do this by using the modules_install target as follows:

$ make modules_install ARCH=powerpc INSTALL_MOD_PATH=/path/to/rfs

Replace the directory /path/to/rfs with the directory containing the RFS for the target board. The installation process copies the files into the directory /path/to/rfs/lib/modules/<kernel version>. If space is a concern, you can remove the kernel modules that won't be used on the target device before this directory is included in the board's root file system.

Kernel modules aren't atomic, which means a kernel module may need another in order to work correctly. This data is stored in the modules.dep file under the $INSTALL_MOD_PATH/lib/modules/<kernel version> directory. A typical entry looks like the following:

kernel/drivers/mtd/mtdblock.ko: kernel/drivers/mtd/mtd_blkdevs.ko

This line says that the mkdblock.ko module depends on the mtd_blkdevs.ko module. Looking at the mtd_blkdevs.ko file shows that no further dependencies exist. Dependency information is close in format to what you find in a make file, so it's easy for a human to read. This information tells you that you need both of these files on the target system if you want to use the features offered by the mtdblock.ko module. This is very helpful when you're removing kernel modules that aren't necessary in order to conserve space.

It's important to note that only the kernel module file (.ko) is necessary to load a module. The additional files, like modules.deps, are a convenience. All you need to load a module on a system is the insmod program, which loads the module into the kernel, and the module itself. However, if your system will be loading arbitrary modules or storage space isn't a concern, you can include all the module files in the target's file system.

Cleaning a kernel build is much like cleaning a regular project: it involves removing the output from prior builds. In the case of Linux, there are several different types of "clean," listed here in order of increasing cleanliness:

The kernel project uses a rolling integration process. This means software checked into project should work and build without error. For a project as big as Linux, this is more that can be done by one person. So, for each major subsystem in the kernel, there is a maintainer who is responsible for reviewing patches. The subsystem maintainers have to ensure that subsystems work, meet the kernel code standards, and build correctly. You can find the canonical list of maintainers in the MAINTAINERS file in the root of the Linux kernel. The maintainers keep a queue of patches at the ready, until Linus Torvalds, the top-level maintainer, sends an e-mail to the maintainers requesting changes, opening what is referred to as the merge window.

The maintainers send their queued changes to Linus, who merges them with the source from the prior release. This process continues for several weeks until the merge window is considered closed. After it's closed, the only changes that are accepted are those that fix defects in the newly accepted code or other major defects. These versions of the Linux kernel are called rc<serial number>, where serial number starts at one and increments upward for every release. A new bug-fix release happens about once a week.

When Linus finds that the code is stable enough—with "stable" meaning that the regression tests are passing and the maintainers working on the kernel agree—Linus issues a new kernel release, and the cycle starts again.

Contributions, big and small, are welcome. Many developers who make small changes to the kernel don't bother contributing their changes, because they view them as not important. If you've made a change that fixes a problem, makes the kernel more stable, or adds a new feature, it will be welcomed by the open source community.

You submit code to maintainers in the form of a patch. Patches show the differences between two files; in order for patches to work, both you and the recipient of the patch need to agree on the basis for the patch. In Linux, patches are made against the latest stable version of the Linux kernel.

To submit a patch, follow these steps:

Will your patch get rejected? Nobody likes rejection. The answer is, "Maybe." Patches are rejected when the maintainer gets several patches that solve the same problem and must select one. Patches are also rejected because they don't solve the problem in a way that the maintainer believes is the best way for the project or when the solution causes regressions or other problems. A maintainer will tell you why your patch wasn't accepted and frequently offer you advice on how to improve it.

Applying a patch means using a program that takes a patch file and makes changes to your source tree as instructed by the patch. The patch file, as explained in the prior section, is nothing more than a diff between your source tree and the prior latest stable version of the kernel. You apply a patch using the patch program like so:

$ cd your-kernel
$ patch -p1 < /path/to/patch.file

-p1 means to ignore the first directory of a patch file. Recall that when the patch was created, it was made by comparing two directories. By dropping the first directory in the patch specification, you're running it relative to your current directory. For example, if the patch was made to the main/init.c program, the patch would contain instructions that some-directory/main/init.c required a change. When you run patch from the root of the kernel source tree, you're currently in the same directory as some-directory; removing that one directory from the specification instructs patch to update the file main/init.c.

Patch doesn't perform atomic updates, in that if there are three changes in three different files, it makes as many changes as possible; patches that don't apply result in error messages, and files that can't be changed are left in their present condition. When you're applying a patch, you can test if it will be successfully applied by doing the following:

$patch -p1 --dry-run < /path/to/patch.file

When you're applying several patch files, the order can be important, because a patch that's making a change in a file may depend on a prior patch being run. When you receive a group of patches, it's critical that you also receive information about the ordering. Many times, patches can be applied in any order, but this isn't always the case; determining the correct order through examination (or experimentation) is impractical.

The next chapter covers real time. Real time, as you'll read doesn't mean fast, it means predictable. Linux systems are referred to as soft-real time system, i.e. systems that can tolerate a certain amount of unpredictable behavior and still be useful. Real time programming is as much a programming practice as it as a technology. The best real time operating system can be mis-used so that it doesn't behave in a predictable manner, no matter what safeguards the designers of the operating system took. Remember, nothing is foolproof because fools possess extra-ordinary cleverness. The chapter also covers some programming techniques that will increase the predictability of the program to further take advantage of the soft real-time capabilities offered by Linux.