CHAPTER 16

Navigating the Linux Filesystem

Several earlier chapters in this book have alluded to the way files are organized on a Linux computer, and have even described some specifics—for instance, that /home is a directory that holds users' home directories. This chapter goes into more detail on this topic. First up is a description of some of the key Linux directories, including their names and purposes. I also describe some special permission bits you can set on files and directories to affect the way Linux treats them. Such attributes can enable you to hide files from view or alter the usual permissions rules in ways that can be helpful for shared user directories.

• Understanding where things go
• Using special permission bits and file features

Understanding Where Things Go

As described in Chapter 6, “Getting to Know the Command Line,” Linux uses a unified directory tree—that is, every partition, removable disk, network file share, and other disk or disk-like storage device is accessible as a directory in a single directory tree (or filesystem). This filesystem is structured—certain directories have certain specific purposes, whether the directories exist as regular subdirectories on one partition or are separate devices that are mounted off the root (/) device. Understanding the purpose of the main directories will help you locate files and avoid making disastrous mistakes. Before delving into those specifics, you should understand the distinction between user files and system files.

User Files vs. System Files

To understand the distinction between user files and system files, you should recall that Linux is a multi-user OS. In principle, a single computer can host thousands of users. Consider such a computer—perhaps a mainframe at a university or a large business or a cloud computing server. The vast majority of this computer's thousands of users will be unfamiliar with the details of Linux system administration—they want only to use their word processors, e-mail clients, and other user applications. Such users don't need to deal with system configuration files. Indeed, giving them access to such files—especially write access—could be disastrous. On a computer with 1,000 users, each of whom can change the system configuration, somebody will make a change that will bring down the computer, whether through ignorance or malice.

Of course, the issues involved in protecting the computer from its users are just a special case of more general user account issues—on that 1,000-user computer, you probably don't want users to be able to read and write each others' files except in limited ways. Thus, as described in Chapter 13, “Understanding Users and Groups” and Chapter 15, “Setting Ownership and Permissions,” you can set permissions on files to prevent unauthorized access.

System files are files that control how the computer operates—they include:

• System startup scripts that launch servers and other important daemons
• Program files, both binary files and scripts
• Program support files, such as fonts and icons
• Configuration files that define how the system works (network configuration settings, disk layout information, and so on)
• Configuration files for most servers and other daemons

  Chapter 9, “Using Programs and Processes,” describes package management.

• Data storage for system programs, such as the database that describes what programs are installed
• System log files, which record normal system activity

Obviously, non-technical users should not be able to alter system files, except perhaps indirectly. (Log files record activities such as login attempts, for instance.) Users must be able to read some types of system files, such as the fonts and icons they use, but some system files should be protected even from read access. Users should not be able to read the /etc/shadow file because it holds encrypted passwords, for instance.

To achieve the goal of restricting ordinary users' access to system files, they are normally owned by root or by another system account that has a more limited purpose. For instance, many server programs rely on their own specific system accounts. System files can then be protected from unwanted access by setting permissions in some appropriate way, depending on their specific needs. Ordinary users are then unable to write to most system files, protecting them from harm. Because root can read and write any file, you must acquire root privileges to perform most system maintenance tasks.

Immediately after installing Linux, most of the files it contains are system files, and most of the directories and subdirectories on a fresh Linux installation are system directories. As described shortly, a few directories, such as /home and /tmp, are set aside for user files, although even these are structured or configured in such a way as to prevent problems with multi-user access.

The distinction between system files and user files exists even on single-user Linux computers such as your personal laptop computer. This may seem strange or frustrating; after all, if you're the only user, and if you have root access, why bother with the distinction between system files and user files? The answer is that it provides a layer of protection against accidental or malicious damage. If a typo, a bug, or malware would, say, delete all the files on the computer, the damage is contained if this action is performed as a normal user rather than as root—a normal user can't delete all the files on the computer. Thus, the distinction between these two account types (and, by extension, these two classes of files) is useful even on a single-user computer.

The Filesystem Hierarchy Standard

Although every Linux distribution has its own unique way of doing certain things, their developers all recognize the need for some standardization in the layout of their directories. For instance, programs should be able to consistently locate key system configuration files in the same places on all distributions. If this weren't the case, programs that rely on such features would become more complex and might not work on all distributions. In order to address this need, the Filesystem Hierarchy Standard (FHS) was created. Even aside from Linux, some Unix-like OSs also follow the FHS to one degree or another.

One important distinction made by the FHS is that between shareable files and unshareable files. Shareable files may be reasonably shared between computers, such as user data files and program binary files. (Of course, you don't need to share such files, but you may do so.) If files are shared, they're normally shared through a Network File System (NFS) server. Unshareable files contain system-specific information, such as configuration files. For instance, you're not likely to want to share a server's configuration file between computers.

A second important distinction made by the FHS is that between static files and variable files. The former don't normally change except through direct intervention by the system administrator. Most program executables are good examples of static files. Variable files may be changed by users, automated scripts, servers, or the like. For instance, users' home directories and mail queues are composed of variable files.

The FHS tries to isolate each directory into one cell of this 2 × 2 (shareable/ unshareable × static/variable) matrix. Figure 16.1 illustrates these relationships. Some directories contain subdirectories in multiple cells, but in these cases, the FHS tries to specify the status of particular subdirectories. For instance, /var is variable, and it contains some shareable and some unshareable subdirectories, as shown in Figure 16.1.

FIGURE 16.1 The FHS attempts to fit each important directory in one cell of a 2 × 2 matrix.

The FHS comes in numbered versions. Version 2.3, the latest version as I write, was released in January 2004. The URL for FHS's official Web page is http://www.pathname.com/fhs/.

Important Directories and Their Contents

The FHS defines the names and purposes of many directories and subdirectories on a Linux system. Table 16.1 summarizes the most important of these. Most of these directories are system directories, the main exceptions being /home, /tmp, /mnt, and /media.

TABLE 16.1 Important Linux directories, according to the FHS

As an ordinary user, you will create most of your files in your home directory, which is normally a subdirectory of /home. You might also access removable media mounted at /media, and perhaps network resources that might be mounted elsewhere. You can use /tmp and certain subdirectories of /var, too, although most users don't need to be explicitly aware of these locations—programs are normally hard-coded to use them for temporary files or for specific types of files, such as incoming e-mail files. As a system administrator, you might manipulate files located in any of these directories; however, for a system administrator, /etc is particularly important, since that's where most system configuration files reside. As you explore your computer with GUI or text-mode utilities, you should keep this directory structure in mind.

Of these directories, several individual directories or collections of them bear special attention:

The configuration directory The /etc directory holds most system configuration files. Previous chapters have referred to several such files, such as /etc/fstab (which defines where partitions are mounted) and /etc/passwd (which is the primary account definition file). Many more exist. Indeed, you'll find subdirectories in /etc to house multiple configuration files for complex subsystems and servers, such as /etc/X11 (for the X Window System) and /etc/samba (for the Samba file server).

Executable directories Program files reside mainly in /sbin, /bin, /usr/sbin, and /usr/bin. (Additional directories can house program files on some systems. Most notably, /usr/local/sbin and /usr/local/bin hold locally compiled programs.)

Library directories Libraries are collections of programming functions that can be useful to many programs. They're stored in separate files to save disk space and RAM when programs run, and enable easy updates to library files without re-installing all the programs that rely on them. In Linux, most libraries reside in /lib and /usr/lib, although some can reside elsewhere (such as /usr/local/lib) on some systems.

If you've administered Windows computers, you should be aware of an important difference between Windows and Linux: In Windows, it's common for a program binary, its configuration files, and all its support files to reside in a single directory tree that belongs to the program, such as C:Program Files SomeProgram. In Linux, by contrast, most of a program's key files are likely to reside in standard locations that are shared with other programs, and to be scattered about. For instance, the program's executable might be in /usr/bin, related libraries in /usr/lib, configuration files in /etc or in users' home directories, and so on. This works well in Linux because Linux's packaging systems, described in Chapter 9, keep track of where a package's many files go, enabling you to easily delete or upgrade a package. The Linux system has the advantage of simplifying the path, which is the list of directories in which program files reside. (Paths also exist for libraries and man pages.) If you're used to looking for files in program-specific locations, though, adjusting to the Linux system can be awkward. The key is to use your package system to identify where a package's files reside; for instance, typing rpm -ql someprogram shows where every file in the someprogram package resides on an RPM-based system.

Using Special Permission Bits and File Features

When you investigate the Linux directory tree, you will encounter certain file types that require special attention. Sometimes you may just want to be aware of how these files are handled, since they deviate from what you might expect based on the information presented in earlier chapters. In other cases, you may need to adjust how you use ls or other commands to deal with these files and directories. These special cases include the “sticky bit,” hiding files from view, obtaining long listings of directories, and using special execute permissions.

Using Sticky Bits

Consider the following commands, typed on a system with a few files and subdirectories laid out in a particular way:

$ whoami
kirk
$ ls -l
total 0
drwxrwxrwx 2 root root 80 Dec 14 17:58 subdir
$ ls -l subdir/
total 2350
-rw-r----- 1 root root 2404268 Dec 14 17:59 f1701.tif

These commands establish the current configuration: The effective user ID is kirk and the current directory has one subdirectory, called subdir, which root owns but to which kirk, like all the system's users, has full read/write access. This subdirectory has one file, f1701.tif, which is owned by root and to which kirk has no access. You can verify that kirk can't write to the file by attempting to do so with the touch command:

$ touch subdir/f1701.tif
touch: cannot touch ′subdir/f1701.tif': Permission denied

This error message verifies that kirk could not write to subdir/f1701.tif. The file, you might think, is safe from tampering. Not so fast! Try this:

$ rm subdir/f1701.tif
$ ls -l subdir/
total 0

The rm command returned no error message, and a subsequent check of subdir verifies that it's now empty—in other words, kirk could delete the file even without write permission to it! This may seem like a bug—after all, if you can't write to a file, you might think you shouldn't be able to delete it. Recall, however, that directories are just a special type of file, one that holds other files' names and pointers to their lower-level data structures. Thus, modifying a file requires write access to the file, but creating or deleting a file requires write access to the directory in which it resides. In this example, kirk has write access to the subdir directory, but not to the f1701.tif file within that directory. Thus, kirk can delete the file but not modify it. This result is not a bug; it's just a counter-intuitive feature.

Although Linux filesystems were designed to work this way, such behavior is not always desirable. The way to create a more intuitive result is to use a sticky bit, which is a special filesystem flag that alters this behavior. With the sticky bit set on a directory, Linux will only permit you to delete a file if you own either it or the containing directory; write permission to the containing directory is not enough. You can set the sticky bit with chown, in either of two ways:

Using an octal code By prefixing the 3-digit octal code described in Chapter 15 with another digit, you can set any of three special permission bits, one of which is the sticky bit. The code for the sticky bit is 1, so you would use an octal code that begins with 1, such as 1755, to set the sticky bit. Specifying a value of 0, as in 0755, removes the sticky bit.

Using a symbolic code Pass the symbolic code t for the world permissions, as in chmod o+t subdir, to set the sticky bit on subdir. You can remove the sticky bit in a similar way by using a minus sign, as in chmod o-t subdir.

Restoring the file and setting the sticky bit enables you to see the effect:

$ ls -l
total 0
drwxrwxrwt 2 root root 80 Dec 14 18:25 subdir
$ ls -l subdir/
total 304
-rw-r--r-- 1 root root 2404268 Dec 14 18:25 f1701.tif
$ rm subdir/f1701.tif
rm: cannot remove ‘subdir/f1701.tif’: Operation not permitted

In this example, although kirk still has full read/write access to subdir, kirk cannot delete another user's files in that directory.

You can identify a directory with the sticky bit set by a small change in the symbolic mode shown by ls -l: The world execute bit is shown as a t rather than an x. In this example, the result is that subdir's permission appears as drwxrwxrwt rather than drwxrwxrwx.

The sticky bit is particularly important for directories that are shared by many users. It's a standard feature on /tmp and /var/tmp, for instance, since many users store temporary files in these directories, and you wouldn't want one user to be able to delete another's temporary files. If you want users who collaborate on a project to be able to write files into each others' home directories, you might want to consider setting the sticky bit on those home directories, or on the subdirectories in which users are sharing files.

Using Special Execute Permissions

As described in Chapter 15, the execute permission bit enables you to identify program files as such. Linux then enables you to run these programs. Such files run using your own credentials, which is generally a good thing—associating running processes with specific users is a key part of Linux's security model. Occasionally, though, programs need to run with elevated privileges. For instance, the passwd program, which sets users' passwords, must run as root to write, and in some cases to read, the configuration files it handles. Thus, if users are to change their own passwords, passwd must have root privileges even when ordinary users run it.

To accomplish this task, two special permission bits exist, similar to the sticky bit described earlier:

Set user ID (SUID) The set user ID (SUID) option tells Linux to run the program with the permissions of whoever owns the file rather than with the permissions of the user who runs the program. For instance, if a file is owned by root and has its SUID bit set, the program runs with root privileges and can therefore read any file on the computer. Some servers and other system programs run this way, which is often called SUID root. SUID programs are indicated by an s in the owner's execute bit position in the permission string, as in rwsr-xr-x.

Set group ID (SGID) The set group ID (SGID) option is similar to the SUID option, but it sets the group of the running program to the group of the file. It's indicated by an s in the group execute bit position in the permission string, as in rwxr-sr-x.

You can set these bits using chmod:

Using an octal code In the leading digit of a 4-digit octal code, set the leading value to 4 to set the SUID bit, to 2 to set the SGID bit, or to 6 to set both bits. For instance, 4755 sets the SUID bit, but not the SGID bit, on an executable file.

Using a symbolic code Use the s symbolic code, in conjunction with u to specify the SGID bit, g to specify the SGID bit, or both to set both bits. For instance, typing chmod u+s myprog sets the SUID bit on myprog, whereas chmod ug-s myprog removes both the SUID bit and the SGID bit.

Ordinarily, you don't need to set or remove these bits; when necessary, the package management program sets these bits correctly when you install or upgrade a program. You might need to alter these bits if they've been mistakenly set or removed on files. In some cases you might want or need to adjust these values on program files that you compile from source code or if you need to modify the way a program works. Be very cautious when doing so, though. If you set the SUID or SGID bit on a garden-variety program, it will run with increased privileges. If the program contains bugs, those bugs will then be able to do more damage. If you accidentally remove these permissions, the results can be just as bad—programs like passwd, sudo, and su all rely on their SUID bits being set, so removing this feature can cause them to stop working.

Hiding Files from View

If you're used to Windows, you may be familiar with the concept of a hidden bit, which hides files from view in file managers, by the Windows DIR command, and in most programs. If you're looking for something analogous in Linux, you won't find it, at least not in the form of a dedicated filesystem feature. Instead, Linux uses a file-naming convention to hide files from view: Most tools, such as ls, hide files and directories from view if their names begin with a dot (.). Thus, ls shows the file afile.txt, but not .afile.txt. Most file managers and dialog boxes that deal with files also hide such dot files, as they're commonly called; however, this practice is not universal.

Many user programs take advantage of this feature to keep their configuration files from cluttering your display. For instance, ˜/.bashrc is a Bash user configuration file, Evolution's configuration files go in the ˜/.evolution directory, and ˜/.fonts.conf holds user-specific font configuration information.

You can view dot files in various ways depending on the program in question. Some GUI tools have a check box you can set in their configuration options to force the program to display such files. At the command line, you can use the -a option to ls:

$ ls -l
total 0
drwxrwxrwt 2 root root 80 Dec 14 18:25 subdir
$ ls -la
total 305
drwxr-xr-x 3 kirk users     104 Dec 14 18:44 .
drwxr-xr-x 3 kirk users     528 Dec 14 18:21 ..
-rw-r--r-- 1 kirk users  309580 Dec 14 18:44 .f1701.tif
drwxrwxrwt 2 root root       80 Dec 14 18:25 subdir

This example shows the hidden file, .f1701.tif, in the current directory. It also shows two hidden directory files. The first, ., refers to the current directory. The second, .., refers to the parent directory.

Note that renaming a file so that it begins with a dot will hide it, but this action will also make the file inaccessible to any program that uses the original filename. That is, if you rename f1701.tif to .f1701.tif, and if another program or file refers to the file as f1701.tif, that reference will no longer work. You must include the leading dot in any reference to the hidden file.

Viewing Directories

Chapter 6 introduced the ls command, including many of its options. One of these deserves elaboration at this point: -d. If you're working in a directory that holds many subdirectories, and if you use a wildcard with ls that matches one or more subdirectories, you may get an unexpected result: The output will show the files in the matched subdirectories, rather than the information on the subdirectories themselves. For instance, starting in a directory with two subdirectories, subdir1 and subdir2:

$ ls -l subdir*
subdir1:
total 304
-rw-r--r-- 1 kirk users 309580 Dec 14 18:54 f1701.tif
subdir2:

total 84
-rw-r--r-- 1 kirk users 86016 Dec 14 18:54 106792c17.doc

If instead you want information on the subdirectories, rather than the contents of those subdirectories, you can include the -d option:

$ ls -ld subdir*
drwxr-xr-x 2 kirk users 80 Dec 14 18:54 subdir1
drwxr-xr-x 2 kirk users 80 Dec 14 18:54 subdir2