Chapter 15. The Boot Process
In this chapter, you learn about making tasks into services that run as your system starts, and making them into services you can start and stop by hand. You also learn about the entire boot process.
After you turn on the power switch, the boot process begins with the computer executing code stored in a chip called the BIOS, or basic input/output system; this process occurs no matter what operating system you have installed. The Linux boot process begins when the code known as the boot loader starts loading the Linux kernel and ends only when the login prompt appears.
As a system administrator, you will use the skills you learn in this chapter to control your system’s services and manage runlevels on your computer. Understanding the management of the system services and states is essential to understanding how Linux works (especially in a multi-user environment) and helps untangle the mysteries of a few of your Ubuntu system’s configuration files. Furthermore, a good knowledge of the cron daemon that handles task scheduling is essential for administrators at all skill levels, so you will want to combine this knowledge with that in Chapter 14, “Automating Tasks and Shell Scripting.”
Running Services at Boot
Although most people consider a computer to be either on or off, in Ubuntu and Linux in general, there are a number of states in between. Known as runlevels, they define what system services are started upon boot. These services are simply applications running in the background that provide some needed function to your system, such as getting information from your mouse and sending it to the display; or a service could monitor the partitions to see whether they have enough free space left on them. Services are typically loaded and run (also referred to as being started) during the boot process, in the same way as Microsoft Windows services are. For a while, Ubuntu used a system known as Upstart instead of the classic and venerable SysVinit; Upstart had a special backward-compatibility layer that can use runlevels in the way that Linux veterans are accustomed to doing for services not otherwise handled by Upstart. Starting in 2015, Ubuntu has switched to another system called systemd. There is more on runlevels, Upstart and systemd later in this chapter.
You can manage nearly every aspect of your computer and how it behaves after booting via configuring and ordering boot scripts and by using various system administration utilities included with Ubuntu. In this chapter, you learn how to work with these boot scripts and system administration utilities. This chapter also offers advice for troubleshooting and fixing problems that might arise with software configuration or the introduction or removal of various types of hardware from your system.
Beginning the Boot Loading Process
Although the actual boot loading mechanism for Linux varies on different hardware platforms (such as the SPARC, Alpha, or PowerPC systems), Intel-based PCs running Ubuntu most often use the same mechanism throughout product lines. This process is traditionally accomplished through a basic input/output system, or BIOS. The BIOS is an application stored in a chip on the motherboard that initializes the hardware on the motherboard (and often the hardware that’s attached to the motherboard). The BIOS gets the system ready to load and run the software that we recognize as the operating system.
As a last step, the BIOS code looks for a special program known as the boot loader or boot code. The instructions in this little bit of code tell the BIOS where the Linux kernel is located, how it should be loaded into memory, and how it should be started.
If all goes well, the BIOS looks for a bootable volume such as a floppy disk, CD-ROM, hard drive, RAM disk, USB drive, or other media. The bootable volume contains a special hexadecimal value written to the volume by the boot loader application (such as Ubuntu’s default, GRUB2) when the boot loader code was first installed in the system’s drives. The BIOS searches volumes in the order established by the BIOS settings (for example, USB first, followed by a DVD-ROM, and then a hard drive) and then boots from the first bootable volume it finds. Modern BIOSs allow considerable flexibility in choosing the device used for booting the system.
Note
If the BIOS detects a hardware problem, the boot process fails, and the BIOS generates a few beeps from the system speaker. These “beep codes” indicate the nature of the problem the BIOS has encountered. The codes vary among manufacturers, and the diagnosis of problems occurring during this phase of the boot process is beyond the scope of this book and does not involve Linux. If you encounter a problem, consult the motherboard manual or contact the manufacturer of the motherboard. Another good source for learning about beep codes is www.computerhope.com/beep.htm.
Next, the BIOS looks on the bootable volume for boot code in the partition boot sector also known as the Master Boot Record (MBR) of the first hard disk. The MBR contains the boot loader code and the partition table—think of it like an index for a book, plus a few comments on how to start reading the book. If the BIOS finds a boot loader, it loads the boot loader code into memory. At that point, the BIOS’s job is completed, and it passes control of the system to the boot loader.
As computing has evolved, the BIOS began to be a limiting factor because some of its limitations were not easy to overcome. Intel was the first company to notice this as they developed their Itanium systems back in the late 1990s. That work eventually became the foundation for the Unified Extensible Firmware Interface (UEFI).
Note
Wikipedia has a nice article that includes the history of UEFI at https://en.wikipedia.org/wiki/Unified_Extensible_Firmware_Interface.
UEFI serves a similar role to BIOS and has replaced BIOS in most modern systems. For most end users, the difference is negligible, except that if you have a UEFI computer, you may need to do some research to install Ubuntu. UEFI firmware often has a BIOS-mode that allows the firmware to work like a traditional BIOS and that is more easily compatible, especially if you are installing Ubuntu as the sole operating system on the computer, rather than as a dual-boot. There were some significant problems early on, but those seem to have lessened as the Linux community learned more about UEFI and made things more easily compatible. To learn more, especially if you are having trouble, see https://help.ubuntu.com/community/UEFI.
One interesting aspect of using UEFI involves hard disk partitioning. When UEFI is used with Windows 8.x or 10, often the partitioning is set up using GPT (GUID Partition Table) rather than the existing MBR (master boot record) standard. You can use GPT with Ubuntu, but if only Ubuntu is to be installed on the drive, it is often easier to reformat the hard disk to use MBR. As with BIOS and UEFI, most people do not need to know or care whether they are using MBR or GPT. To learn more, see http://www.howtogeek.com/193669/whats-the-difference-between-gpt-and-mbr-when-partitioning-a-drive/.
The boot loader locates the Linux kernel on the disk and loads it into memory. After that task is completed, the boot loader passes control of the system to the Linux kernel. You can see how one process builds on another in an approach that enables many different operating systems to work with the same hardware.
Note
Linux is very flexible and can be booted from multiple images on a CD-ROM, over a network using PXE (pronounced “pixie”) or NetBoot, or on a headless server with the console display sent over a serial or network connection. Work is even underway to create a special Linux BIOS at www.coreboot.org/ that will expedite the boot process because Linux does not need many of the services offered by the typical BIOS.
This kind of flexibility enables Linux to be used in a variety of ways, such as remote servers or diskless workstations, which are not generally seen in personal home use.
Loading the Linux Kernel
In a general sense, the kernel manages the system resources. As the user, you do not often interact with the kernel, but instead you interact with the applications that you are using. Linux refers to each application as a process, and the kernel assigns each process a number called a process ID (PID). Traditionally, the Linux kernel loads and runs a process named init, which is also known as the “ancestor of all processes” because it starts every subsequent process. The traditional init system was SysVinit. It has been replaced by newer options. Until recently, Ubuntu replaced init with Upstart, which was written by Ubuntu developers and made available for any distribution to use. Upstart has now been replaced by systemd as of Ubuntu 15.04. This chapter walks through the traditional SysVinit method first and then provides details on Upstart and systemd.
This next step of the boot process traditionally begins with a message that the Linux kernel is loading, and a series of messages that are printed to the screen, giving you the status of each command. A failure should display an error message. The quiet option may be passed to the kernel at boot time to suppress many of these messages. Ubuntu does not display these messages by default, but instead uses a boot process created by the Fedora/Red Hat developers called Plymouth that is fast and incorporates a beautiful boot screen.
If the boot process were halted at this point, the system would just sit idle and the screen would be blank. To make the system useful for users, you need to start the system services. Those services are some of the applications that enable you to interact with the system.
System Services and Runlevels
The init command traditionally boots a Linux system to a specific system state, commonly referred to as its runlevel.
Runlevels determine which of the many available system services are started, as well as in which order they start. A special runlevel is used to stop the system, and a special runlevel is used for system maintenance. As you will see, there are other runlevels for special purposes.
You traditionally use runlevels to manage the system services running on a Linux computer. All these special files and scripts are set up during installation, but you can change and control them manually.
With the integration of systemd, much of the interest and concern about runlevels is unnecessary. This is because systemd replaces much of the functionality previously performed by runlevels. For example, you don’t even see the runlevel concept used anymore, but instead you hear generic terms like booting into the default.target, multi-user.target, graphical.target, or reboot.target. This information remains in the book for historic context, but most users can simply skim the runlevel sections.
Runlevel Definitions
The runlevels are defined in a traditional Linux system in /etc/init.d. Some distributions use the traditional /etc/inittab file to manage boot services. Ubuntu has not used this for several years. Because it is not standard in Ubuntu, this book does not cover /etc/inittab.
Each runlevel tells the init command what services to start or stop. Although runlevels might all have custom definitions, Ubuntu has adopted some standards:
Runlevel 0—Known as “halt,” this runlevel shuts down the system.
Runlevel 1—This is a special runlevel, defined as “single,” which boots Ubuntu to a root access shell prompt where only the root user may log in. It has networking, X, and multi-user access turned off. This is the maintenance or rescue mode. It enables the system administrator to perform work on the system, make backups, or repair configuration or other files.
Runlevels 2–5—These runlevels aren’t used in Ubuntu in any way that distinguishes them from each other, but are often used in other Linux distributions.
Runlevel 6—This runlevel reboots the system.
Runlevel 1 (also known as single-user mode or maintenance mode) is most commonly used to repair file systems and change the root password on a system when the password has been forgotten. Trespassers with physical access to the machine can also use runlevel 1 to access your system.
Never forget that uncontrolled physical access is a virtual guarantee of access to your data by an intruder.
Booting into the Default Runlevel
Traditionally, Ubuntu boots into runlevel 5 by default, which means it starts the system as normal and leaves you inside the X Window System looking at the graphical login prompt. It knows what runlevel 5 needs to load by looking in the rc*.d directories in /etc. Ubuntu contains directories for rc0.d through to rc5.d and rcS.d.
Assuming that the value is 1, the rc script executes all the scripts under the /etc/rc.1 directory and then launches the graphical login.
If Ubuntu is booted to runlevel 1, for example, scripts beginning with the letter K followed by scripts beginning with the letter S under the /etc/rc1.d directory are then executed:
matthew@seymour:~$ ls /etc/rc1.d/
K10jackd K20rsync K20vboxdrv K80cups S70pppd-dns
K15pulseaudio K20saned K20winbind README S90single
K20acpi-support K20saslauthd K74bluetooth S30killprocs
K20kerneloops K20speech-dispatcher K77ntp S70dns-clean
These scripts, as with all scripts in the rc*.d directories, are actually symbolic links to system service scripts that reside in the /etc/init.d directory.
The rc1.d links are prefaced with a letter and number, such as K15 or S10. The K or S in these prefixes indicate whether a particular service should be killed (K) or started (S) and pass a value of stop or start to the appropriate /etc/init.d script. The number in the prefix executes the specific /etc/init.d script in a particular order. The symlinks have numbers to delineate the order in which they are started. Nothing is sacred about a specific number, but some services need to be running before others are started. You would not want your Ubuntu system to attempt, for example, to mount a remote Network File System (NFS) volume without first starting networking and NFS services.
Understanding init Scripts and the Final Stage of Initialization
Each /etc/init.d script, or init script, contains logic that determines what to do when receiving a start or stop value. The logic might be a simple switch statement for execution, as in this example:
case "$1" in
start)
start
;;
stop)
stop
;;
restart)
restart
;;
reload)
reload
;;
status)
rhstatus
;;
condrestart)
[ -f /var/lock/subsys/smb ] && restart || :
;;
*)
echo $"Usage: $0 {start|stop|restart|status|condrestart}"
exit 1
esac
Although you can use the scripts to customize the way that the system runs from power-on, absent the replacement of the kernel, this script approach also means that you do not have to halt the system in total to start, stop, upgrade, or install new services.
Note that not all scripts use this approach and that other messages might be passed to the service script, such as restart, reload, or status. Also, not all scripts respond to the same set of messages (with the exception of start and stop, which they all have to accept by convention) because each service might require special commands.
After all the system scripts have been run, your system is configured and all the necessary system services have been started. If you are using a runlevel other than 5, the final act of the init process is to launch the user shell—bash, tcsh, zsh, or any of the many command shells available. The shell launches and you see a login prompt on the screen.
Controlling Services at Boot with Administrative Tools
You can configure what services run at startup from the Dash with a search for Startup Applications (shown in Figure 15.1). Here Ubuntu lists all the services that you can have automatically start at boot time. They are usually all enabled by default, but you can uncheck the ones you don’t want and click OK. It is not recommended that you disable services randomly “to make things go faster.” Some services might be vital for the continuing operation of your computer, such as the graphical login manager and the system communication bus.
FIGURE 15.1 You can enable and disable Ubuntu’s boot services by toggling the check boxes in the Services dialog.
Changing Runlevels
After making changes to system services and runlevels, you can use the telinit command to change runlevels on-the-fly on a running Ubuntu system. Changing runlevels this way enables system administrators to alter selected parts of a running system to make changes to the services or to put changes into effect that have already been made (such as reassignment of network addresses for a networking interface).
For example, in the past a system administrator could quickly change the system to maintenance or single-user mode by using the telinit command with its S option like this:
matthew@seymour:~$ sudo telinit S
Today, the same thing would be done using this systemd command:
matthew@seymour:~$ sudo systemctl rescue
The telinit command uses the init command to change runlevels and shut down currently running services. However, under systemd, telinit is deprecated.
After booting to single-user mode, you used to then return to multi-user mode, like this:
matthew@seymour:~$ sudo telinit 2
Today, the same thing would be done using this systemd command:
matthew@seymour:~$ systemctl default
Linux is full of shortcuts:
If you don’t want to use the systemctl default command, you can just click Ctrl+D.
If you exit the single-user shell by typing exit at the prompt, you go back to the default runlevel without worrying about using telinit or systemctl.
Troubleshooting Runlevel Problems
Reordering or changing system services during a particular runlevel is rarely necessary when using Ubuntu unless some disaster occurs. But system administrators should have a basic understanding of how Linux boots and how services are controlled in order to perform troubleshooting or to diagnose problems. By using additional utilities such as the dmesg | less command to read kernel output after booting or by examining system logging with cat /var/log/messages | less, it is possible to gain a bit more detail about what is going on when faced with troublesome drivers or service failure.
Starting and Stopping Services Manually
If you change a configuration file for a system service, it is usually necessary to stop and restart the service to make it read the new configuration. If you are reconfiguring the X server, it is often convenient to change from runlevel 5 to runlevel 1 to make testing easier and then switch back to runlevel 5 to reenable the graphical login. If a service is improperly configured, it is easier to stop and restart it until you have it configured correctly than it is to reboot the entire machine.
The traditional way to manage a service (as root) is to call the service’s /etc/init.d name on the command line with an appropriate keyword, such as start, status, restart, or stop. For example, to start the Apache web server, call the /etc/init.d/apache2 script like this:
matthew@seymour:~$ sudo /etc/init.d/apache2 start
Starting apache 2.2 web server [ OK ]
The script executes the proper programs and reports the status of it/them. Stopping services is equally easy, using the stop keyword.
Using Upstart
Starting in 2015, Ubuntu has switched to systemd instead of Upstart. This content is being retained in the book for users of LTS versions like 14.04 LTS that are still supported, but for anyone doing a new installation, you should skip this section and move immediately to the next section, systemd. If you are upgrading from an earlier release of Ubuntu, you may benefit from reading https://wiki.ubuntu.com/SystemdForUpstartUsers.
Upstart was originally developed for Ubuntu. It is an event-based replacement for the /sbin/init daemon and System-V init system. It handles starting of tasks and services during boot and stops them during shutdown. It also supervises them while the system is running and is intended to become a way to have tasks and services start or stop automatically based on specific events that happen rather than having to call a script manually to start or stop them. This is a big change and a big deal because it will make the overall system much more flexible, configurable, and responsive to conditions.
Basically, in Upstart, tasks and services are started and stopped by events. Events are generated as other tasks and services are started and stopped and may be received from any other process on the system. Services may be respawned if they die unexpectedly and communication with the init daemon occurs over D-Bus. Planned features that have not yet been implemented include the ability for events to be created at timed intervals or at scheduled times or as files or directories are changed and the creation of user services that users can start and stop themselves. This means that eventually Upstart jobs could even replace the venerable cron and do things previously impossible on any UNIX or Linux version.
For now, Upstart is being used for many jobs including boot and shutdown, but not all services have Upstart jobs written. Those that exist reside as expected in /etc/init and are easily accessible in the same basic manner as init jobs. If an Upstart job exists and you try to manage it using the traditional method, you get a message like this:
matthew@seymour:~$ sudo /etc/init.d/ufw stop
[sudo] password for matthew:
Rather than invoking init scripts through /etc/init.d, use the service(8)
utility, e.g. service ufw stop
Since the script you are attempting to invoke has been converted to an
Upstart job, you may also use the stop(8) utility, e.g. stop ufw
ufw stop/waiting
In this case, to start or stop Ubuntu’s Uncomplicated Firewall (UFW), you just enter sudo start ufw or sudo stop ufw. Simple. If you want to learn more about Upstart, start with http://upstart.ubuntu.com/. There is also an excellent cookbook filled with Upstart recipes at http://upstart.ubuntu.com/cookbook/.
Read the systemd section that follows this one before you dig too deeply into Upstart.
systemd
Starting 2015, Upstart has been replaced with systemd. systemd was developed at the same time as Upstart and the two have been competing for the title of “init replacement of the future.” systemd won. There isn’t much point in learning a ton about Upstart, other than what you need for the moment. The future lies with systemd.
To interact with systemd, you typically use the systemctl command. The main interactions you are likely to have with systemd involve starting and stopping services. Table 15.1 shows commands related to services.
There are a few standard commands that are replaced, like the ones in Table 15.2.
Services are defined in systemd Unit files, which end with .service. Many examples of these are found in /lib/systemd/system.
There is much more to learn about systemd than is appropriate for a book with such a wide range of topics to cover, especially as few people need to know it. See the official documentation listed in References to learn more, if you find yourself in need.
Boot Repair
Sometimes, such as when you install both Windows and Ubuntu on the same hard drive, boot problems can develop. Boot Repair is a simple GUI tool to fix those problems. Typically, it just reinstalls GRUB2, but using the program is a much easier solution for many users. The program is not yet in the Ubuntu repositories, although plans for including it have been discussed. In the meanwhile, should you need it, take a look at https://help.ubuntu.com/community/Boot-Repair or the official documentation at http://sourceforge.net/projects/boot-repair/.
References
/usr/src/linux/init/main.c—This file is found on your computer after you install the kernel source code; it’s the best place to learn about how Linux boots. Fascinating reading, really. Get it from the source.
https://help.ubuntu.com/community/Grub2—Ubuntu community documentation for GRUB2.
https://wiki.freedesktop.org/www/Software/systemd/—The official systemd documentation.