In this chapter you will learn

This is one of the most important concepts that make Linux especially flexible and powerful: everything is a file. That is, everything can be the source of a data stream, the target of a data stream, or in many cases both. In this chapter you will explore what “everything is a file” really means and learn to use that to advantage as a SysAdmin.

The whole point with "everything is a file" is ... the fact that you can use common tools to operate on different things.

—Linus Torvalds in an email.

Here is a trick question for you. Which of the following are files?

To Unix and Linux, they are all files and that is one of the most amazing concepts in the history of computing. It makes possible some very simple yet powerful methods for performing many administrative tasks that might otherwise be extremely difficult or impossible.

Linux handles almost everything as a file. This has some interesting and amazing implications. This concept makes it possible to copy an entire hard drive, boot record included, because the entire hard drive is a file, just as are the individual partitions.

“Everything is a file” is possible because all devices are implemented by Linux as these things called device files. Device files are not device drivers, rather they are gateways to devices that are exposed to the user.

Device files are technically known as device special files.¹ Device files are employed to provide the operating system and the users an interface to the devices that they represent. All Linux device files are located in the /dev directory, which is an integral part of the root (/) filesystem because they must be available to the operating system during early stages of the boot process – before other filesystems are mounted.

udev is designed to simplify this problem by creating entries in /dev only for those devices that actually currently exist at boot time or which have a high probability of actually existing on the host. This significantly reduces the total number of device files required.

In addition, udev assigns names to devices when they are plugged into the system, such as USB storage and printers, and other non-USB types of devices as well. In fact, udev treats all devices as plug and play (PnP), even at boot time. This makes dealing with devices consistent at all times, whether at boot time or when they are hot-plugged later. It is not necessary for us as SysAdmins to do anything else for the device files to be created. The Linux kernel takes care of everything. It is only possible to mount the partition in order to access its contents after the device file such as /dev/sdb1 has been created.

Kernel developer and maintainer Greg Kroah-Hartman, one of the creators of udev, has written a paper² that provides some insight into the details of udev and how it is supposed to work. Note that udev has matured since the article was written and some things have changed, such as the udev rule locations and structure. Regardless, this paper provides some deep and important insight into udev and current device naming strategies which I’ll attempt to summarize in this chapter.

Let's look at the data flow of a typical command to visualize how device special files work. Figure 3-1 illustrates a simplified data flow for a simple command. Issuing the # cat /etc/resolv.conf command from a GUI terminal emulator such as Konsole or xterm causes the resolv.conf file to be read from the disk with the disk device driver handling the device-specific functions such as locating the file on the hard drive and reading it. The data is passed through the device file and then from the command to the device file and device driver for pseudo-terminal 6 where it is displayed in the terminal session.

Of course the output of the cat command could have been redirected to a file in the following manner, cat /etc/resolv.conf > /etc/resolv.bak, in order to create a backup of the file. In that case the data flow on the left side of Figure 3-1 would remain the same, while the data flow on the right would be through the /dev/sda2 device file, the hard drive device driver, and then back onto the hard drive in the /etc directory as the new file, resolv.bak.

These device special files make it very easy to use standard streams (STDIO) and redirection to access any and every device on a Linux or Unix computer. They provide a consistent and easy to access interface to every device. Simply directing a data stream to a device file sends the data to that device.

One of the most important things to remember about these device special files is that they are not device drivers. They are most accurately described as portals or gateways to the device drivers. Data is passed from an application or the operating system to the device file which then passes it to the device driver which then sends it to the physical device.

By using these device files which are separate from the device drivers, it is possible for users and programs to have a consistent interface to every device on the host computer. This is how common tools can be used to operate on different things as Linus says.

The device drivers are still responsible for dealing with the unique requirements of each physical device. That is, however, outside the scope of this book.

Device files can be classified in at least two ways. The first and most commonly used classification is that of the type of data stream commonly associated with the device. For example, tty and serial devices are considered to be character based because the data stream is transferred and handled one character or byte at a time. Block-type devices such as hard drives transfer data in blocks, typically a multiple of 256 bytes.

Let's take a look at the /dev/directory and some of the devices in it.

The pruned listing of device files shown in Experiment 3-1 are just a few of the ones in the /dev/directory on my StudentVM1 virtual machine. The ones on your VM should be very similar if not identical. They represent disk and tty type devices among many others. Notice the leftmost character of each line in the output. The ones that have a “b” are block-type devices and the ones that begin with “c” are character devices.

The more detailed and explicit way to identify device files is using the device major and minor numbers. The disk devices have a major number of 8 which designates them as SCSI block devices. Note that all parallel ATA (PATA)⁴ and serial ATA (SATA)⁵ hard drives and SSDs have been managed by the SCSI subsystem because the old ATA subsystem was deemed many years ago as not maintainable due to the poor quality of its code. As a result, hard drives that would previously be designated as “hd[a-z]” are now referred to as “sd[a-z]”.

You can probably infer the pattern of disk drive minor numbers in the small sample shown earlier. Minor numbers 0, 16, 32, and so on up through 240 are the whole disk numbers. So major/minor 8/16 represents the whole disk /dev/sdb and 8/17 is the device file for the first partition, /dev/sdb1. Numbers 8/34 would be /dev/sdc2.

The tty device files in the preceding list are numbered a bit more simply from tty0 through tty63. I find the number of tty devices a little incongruous because the whole point of the new udev system is to create device files for only those devices that actually exist; I am not sure why it is being done this way. The device files on your host should have a timestamp that is the same as the last boot time.

The Linux Allocated Devices⁶ file at Kernel.org is the official registry of device types and major and minor number allocations. It can help you understand the major/minor numbers for all currently defined devices.

Let's take a few minutes now and have some fun with some of these device files. We will perform a couple fun experiments that illustrate the power and flexibility of the Linux device files.

Most Linux distributions have multiple virtual consoles, 1 through 7, that can be used to log in to a local console session with a shell interface. These can be accessed using the key combinations HostKey-F1 for console 1, HostKey-F2 for console 2, and so on. Virtual consoles were introduced in Volume 1, Chapter 7, of this course. The default HostKey is the right Control key, but I have reconfigured mine to be the left Win key, a.k.a. the super key, because I find it easier. You can change the default HostKey with the VirtualBox manager.

Another interesting experiment is to print a file directly to the printer using the cat command. If you do not have a printer attached to your physical host that is available to the VM, you can skip Experiment 3-3.

The /dev directory contains some very interesting device files that are portals to hardware that one does not normally think of as a device like a hard drive or display. For one example, system memory – RAM – is not something that is normally considered as a “device,” yet /dev/mem is the device special file through which direct access to memory can be achieved.

The dd command provides significantly more control than simply using the cat command to dump all of memory, which I have also tried. The dd command provides the ability to specify how much data is read from /dev/mem and would also allow me to specify the point at which to start reading data from memory. Although some memory was read using the cat command, the kernel eventually responded with the error in Figure 3-2.

You can also log in as a non-root user, student, and try this command. You will get an error message because the memory you are trying to access does not belong to your user. This is a memory protection feature of Linux that keeps other users from reading or writing memory that does not belong to them.

Many types of malware depend upon privilege escalation to allow them to read the contents of memory that they would not normally be able to access. This allows the malware to find and steal personal data such as account numbers, user ID, and stored passwords. Fortunately, Linux protects against memory access by non-root users. It also protects against privilege escalation. But even Linux security is not perfect. It is important to install security patches to protect against vulnerabilities that allow privilege escalation. You should also be aware of human factors such as the tendency people have to write down their passwords, but that is all another book.⁷

You can now see that memory is also considered to be a file and can be treated as such using the memory device file.

There are some other very interesting device files in /dev. The device special files null, zero, random, and urandom are not associated with any physical devices. These device files provide sources of zeros, nulls, and random numbers.

The null device /dev/null can be used as a target for the redirection of output from shell commands or programs so that they are not displayed on the terminal.

The /dev/random and /dev/urandom devices are both useful as data stream sources. As their names imply, they both produce essentially random output – not just numbers but any and all byte combinations. The /dev/urandom device produces a deterministic⁸ stream of random output and is very fast while /dev/random produces a non-deterministic⁹ stream but is slower.

The /dev/random device file produces non-deterministic random output, but it produces output more slowly. This output is not determined by an algorithm that is dependent only upon the previous number that was generated, but it is generated in response to keystrokes and mouse movements. This method makes it far more difficult to duplicate a specific series of random numbers. Use the cat command to view some of the output from the /dev/random device file. Try moving the mouse to see how it affects the output.

The random data generated from /dev/random and /dev/urandom, regardless of how it is read from those devices, is usually redirected to a file on some storage media or to STDIN of another program. Random data seldom needs to be viewed by the SysAdmin, developer, or the end user. But it does make a good demonstration for this experiment.

As its name implies, the /dev/zero device file produces an unending string of zeros as output. Note that these are octal zeros and not the ASCII character zero (0).

Consider, for example, the simple task of making a backup of the master boot record (MBR) of a hard drive. I have had, on occasion, needed to restore or recreate my MBR, particularly the partition table. Recreating it from scratch is very difficult. Restoring it from a saved file is easy. So let's back up the boot record of the hard drive.

Note that all of the experiments in this section must be performed as root.

If the MBR were damaged, it would be necessary to boot to a rescue disk and use the command in Figure 3-3 which would perform the reverse operation of the preceding one. Notice that it is not necessary to specify the block size and block count as in the first command because the dd command will simply copy the backup file to the first sector of the hard drive and stop when it reaches the end of the source file.

So now that you have performed a backup of the boot record of your hard drive and verified the contents of that backup, let's move to a safer environment to destroy the boot record and then restore it.

The implications of “everything is a file” are far-reaching and much greater than can be listed here. You have already seen some examples in the preceding experiments. But here is a short list that encompasses those and more:

There are so many possibilities here that any list can really only scratch the surface. I am sure that you have – or will – figure out many ways to use this tenet of the Linux Philosophy far more creatively than I have discussed here.

It is all part of a filesystem. Everything on a Linux computer is accessible as a file in the filesystem space. The whole point of this is to be able to use common tools to operate on different things – common tools such as the standard GNU/Linux utilities and commands that work on files will also work on devices – because, in Linux, they are files.

Perform the following exercises to complete this chapter: