Each of the three access digits represents the permissions for a different class of users. The first digit represents the permissions for the file’s owner, the second digit represents permissions for users in the file’s group, and the final digit represents the permissions for all other users. Each octal digit is made up of three bits, which represent read permission, write permission, and execute permission, from most significant to least significant bit. The term world permissions is commonly used to refer to the permission given to all three classes of users.

Let’s try to make the previous paragraph a little more concrete through some examples. Linux’s chmod command allows the user to specify an access mode in octal and then applies that mode to one or more files. If we have a file, somefile, which we would like to allow only the owner to write to but any user (including the owner) to read from, we would use mode 0644 (remember, this is in octal). The leading 6 is 110 in binary, which indicates that the type of user to which it refers (in this case, the owner) is given both read and write permission; the 4s are 010 binary, giving the other types of users (group and other) only read permissions.

$ chmod 0644 somefile
$ ls -l somefile
-rw-r--r--   1 ewt       devel       31 Feb 15 15:12 somefile

If we wanted to allow any member of group devel to write to the file, we would use mode 0664 instead.

$ chmod 0664 somefile
$ ls -l somefile
-rw-rw-r--   1 ewt      devel       31 Feb 15 15:12 somefile

If somefile is a shell script (programs that use #! at their beginning to specify a command interpreter) we want to execute, we must tell the system that the file is executable by turning on the execute bit—in this case, we are allowing the owner to read, write, and execute the file and members of group devel to read and execute the file. Other users may not manipulate the file in any way.

$ chmod 0750 somefile
$ ls -l somefile
-rwxr-x---   1 ewt      devel       31 Feb 15 15:12 somefile

Directories use the same set of access bits as normal files, but with slightly different semantics. Read permissions allow a process to access the directory itself, which lets a user list the contents of a directory. Write permissions allow a process to create new files in the directory and delete existing ones. The execute bit does not translate as well, however (what does it mean to execute a directory?). It allows a process to search a directory, which means it can access a file in that directory, as long as it knows the name of that file.

Most system directories on Linux machines have 0755 permissions and are owned by the root user. This lets all users on the system list the files in a directory and access those files by name, but restricts writing in that directory to the root user. Anonymous ftp sites that allow any person to submit files but do not want to let people download them until an administrator has looked at the contents of the files, normally set their incoming file directories to 0772. That allows all users to create new files in the directory without being allowed either to see the contents of the directory or access files in it.

More information on file access permissions may be found in any introductory Linux or Unix book [Sobell, 2002] [Welsh, 1996].

The file permission modifier digit is also a bitmask, whose values represent setuid, setgid, and the sticky bit. If the setuid bit is set for an executable file, the process’s effective user ID is set to the owner of the file when the program is executed (see page 111 for information on why this is useful). The setgid bit behaves the same way, but sets the effective group ID to the file’s group. The setuid bit has no meaning for files that are not executable, but if the setgid bit is set for a nonexecutable file, any locking done on the file is mandatory rather than advisory.^[8] Under Linux, the setuid and setgid bits are ignored for shell scripts because setuid scripts tend to be insecure.

Neither setuid nor setgid bits have any obvious meaning for directories. The setuid bit actually has no semantics when it is set on a directory. If a directory’s setgid bit is set, all new files created in that directory are owned by the same group that owns the directory itself. This makes it easier to use directories for collaborative work among users.

The sticky bit, the least significant bit in the file permission modifier digit, has an interesting history behind its name. Older Unix implementations had to load an entire program into memory before they could begin executing it. This meant big programs had long startup times, which could get quite annoying. If a program had the sticky bit set, the operating system would attempt to leave the program “stuck” in memory for as long as possible, even when the program was not running, reducing the startup time for those programs. Although this was a bit of a kludge, it worked reasonably well for commonly used programs, such as the C compiler. Modern Unix implementations, including Linux, use demand loading to run programs that load the program piece by piece, making the sticky bit unnecessary, and so Linux ignores the sticky bit for regular files.

The sticky bit is still used for directories. Usually, any user with write permissions to a directory can erase any file in that directory. If a directory’s sticky bit is set, however, files may be removed only by the user who owns the file and the root user. This behavior is handy for directories that are repositories for files created by a wide variety of users, such as /tmp.

The final section of a file’s mode specifies the file’s type. It is contained in the high-order octal digits of the mode and is not a bitmask. Instead, the value of those digits equates to a specific file type (04 indicates a directory; 06 indicates a block device). A file’s type is set when the file is created. It can never be changed except by removing the file.

The include file <sys/stat.h> provides symbolic constants for all of the access bits, which can make code more readable. Both Linux and Unix users usually become comfortable with the octal representation of file modes, however, so it is common for programs to use the octal values directly. Table 11.1 lists the symbolic names used for both file access permissions and file permission modifiers.

The upper four bits of a file mode specify the file’s type. Table 11.2 lists the constants that relate to the file’s type. Bitwise AND’ing any of these constants with a file’s mode yields non-0 if the bit is set.

The following macros take a file mode as an argument and return true or false:

The permissions given to newly created files depend on both a system’s setup and an individual user’s preferences. To relieve individual programs of the need to guess the permissions to use for a file, the system allows users to turn off particular permissions for newly created files (and directories, which are just special files). Every process has a umask, which specifies the permission bits to turn off when files are created. This allows a process to specify fairly liberal permissions (usually, world read and write permissions) and end up with the permissions the user would like. If the file is particularly sensitive, the creating process can specify more restrictive permissions than normal, because the umask never results in less restrictive permissions, only in more restrictive permissions.

The process’s current umask is set by the umask() system call.

#include <sys/stat.h>

int umask(int newmask);

The old umask is returned, and the process’s umask is set to the new value. Only read, write, and execute permissions may be specified for the file—you cannot use the umask to prevent the setuid, setgid, or sticky bits from being set. The umask command present in most shells allows the user to set the umask for the shell itself and its subsequent child processes.

As an example, the touch command creates new files with 0666 (world read and write) permissions. Because the user rarely wants this, he could force the touch command to turn off world and group write permissions for a file with a umask of 022, as shown by this example:

$ umask 022
$ touch foo
$ ls -l foo
-rw-r--r--   1 ewt     ewt        0 Feb 24 21:24 foo

If he prefers group write permissions, he can use a umask of 002 instead.

$ umask 002
$ touch foo2
$ ls -l foo2
-rw-rw-r--   1 ewt     ewt        0 Feb 24 21:24 foo2

If he wants all his files to be accessible only by himself, a 077 umask will accomplish the task.

$ umask 077
$ touch foo3
$ ls -l foo3
-rw-------   1 ewt     ewt         0 Feb 24 21:26 foo3

The process’s umask affects the open(), creat(), mknod(), and mkdir() system calls.

When a process gains access to a file (usually called opening the file), the kernel returns a file descriptor that the process uses to perform subsequent operations on the file. File descriptors are small, positive integers, which serve as indices into an array of open files the kernel maintains for each process.

The first three file descriptors for a process (0, 1, and 2) have standard usages. The first, 0, is known as standard input (stdin) and is where programs should take their interactive input from. File descriptor 1 is called standard output (stdout), and most output from the program should be directed there. Errors should be sent to standard error (stderr), which is file descriptor 2. The standard C library follows these rules, so gets() and printf() use stdin and stdout, respectively, and these conventions allow shells to properly redirect a process’s input and output.

The <unistd.h> header file provides the STDIN_FILENO, STDOUT_FILENO, and STDERR_FILENO macros, which evaluate to the stdin, stdout, and stderr file descriptors, respectively. Using these symbolic names can make code slightly more readable.

Many of the file operations that manipulate a file’s inode are available in two forms. The first form takes a file name as an argument. The kernel uses that argument to look up the file’s inode and performs the appropriate operation on the inode (this usually includes following symlinks). The second form takes a file descriptor as an argument and performs the operation on the inode it refers to. The two sets of system calls use similar names, with the system calls that expect a file descriptor argument prefixed with the letter f. For example, the chmod() system call changes the access permissions for the file referred to by the passed file name; fchmod() sets the access permissions for the file referred to by the specified file descriptor.

To make the rest of this discussion a bit less verbose, we present both versions of the system calls when they exist but discuss only the first version (which uses a file name).

One of the few operations that is the same for all types of files is closing the file. Here is how to close a file:

#include <unistd.h>

int close(int fd);

This is obviously a pretty basic operation. However, there is one important thing to remember about closing files—it could fail. Some systems (most notably, networked file systems such as NFS) do not try to store the final piece of written data in the file system until the file is closed. If that storage operation fails (the remote host may have crashed), then the close() returns an error. If your application is writing data but does not use synchronous writes (see the discussion of O_SYNC in the next section), you should always check the results of file closures. If close() fails, the updated file is corrupted in some unpredictable fashion! Luckily, this happens extremely rarely.

Although Linux provides many types of files, regular files are by far the most commonly used. Programs, configuration files, and data files all fall under that heading, and most applications do not (explicitly) use any other file type. There are two ways of opening files that have associated file names:

#include <fcntl.h>
#include <unistd.h>

int open(char * pathname, int flags, mode_t mode);
int creat(char * pathname, mode_t mode);

The open() function returns a file descriptor that references the file pathname. If the return value is less than 0, an error has occurred (as always, errno contains the error code). The flags argument describes the type of access the calling process wants, and also controls various attributes of how the file is opened and manipulated. An access mode must always be provided and is one of O_RDONLY, O_RDWR, and O_WRONLY, which request read-only, read-write, and write-only access, respectively. One or more of the following values may be bitwise OR’ed with the access mode to control other file semantics.

The mode parameter specifies the access permissions for the file if it is being created, and it is modified by the process’s current umask. If O_CREAT is not specified, the mode is ignored.

The creat() function is exactly equivalent to

open(pathname, O_CREAT | O_WRONLY | O_TRUNC, mode)

We do not use creat() in this book because we find open() easier to read and to understand.^[11]

Although there are a few ways to read from and write to files, only the simplest is discussed here.^[12] Reading and writing are nearly identical, so we discuss them simultaneously.

#include <unistd.h>

size_t read(int fd, void * buf, size_t length);
size_t write(int fd, const void * buf, size_t length);

Both functions take a file descriptor fd, a pointer to a buffer buf, and the length of that buffer. read() reads from the file descriptor and places the data read into the passed buffer; write() writes length bytes from the buffer to the file. Both functions return the number of bytes transferred, or -1 on an error (which implies no bytes were read or stored).

Now that we have covered these system calls, here is a simple example that creates the file hw in the current directory and writes Hello World! into it:

 1: /* hwwrite.c */
 2:
 3: #include <errno.h>
 4: #include <fcntl.h>
 5: #include <stdio.h>
 6: #include <stdlib.h>
 7: #include <unistd.h>
 8:
 9: int main(void) {
10:     int fd;
11:
12:     /* open the file, creating it if it's not there, and removing
13:        its contents if it is there */
14:     if ((fd = open("hw", O_TRUNC | O_CREAT | O_WRONLY, 0644)) < 0) {
15:         perror("open"),
16:         exit(1);
17:     }
18:
19:     /* the magic number of 13 is the number of characters which will
20:        be written */
21:     if (write(fd, "Hello World!
", 13) != 13) {
22:         perror("write");
23:         exit(1);
24:     }
25:
26:     close(fd);
27:
28:     return 0;
29: }

Here is what happens when we run hwwrite:

$ cat hw
cat: hw: No such file or directory
$ ./hwwrite
$ cat hw
Hello World!
$

Changing this function to read from a file is a simple matter of changing the open() to

open("hw", O_RDONLY);

and changing the write() of a static string to a read() into a buffer.

Unix files can be divided into two catgories: seekable and nonseekable.^[13] Nonseekable files are first-in/first-out channels that do not support random reads or writes, and data cannot be reread or overwritten. Seekable files allow the reads and the writes to occur anywhere in the file. Pipes and sockets are nonseekable files; block devices and regular files are seekable.

As FIFOs are nonseekable files, it is obvious where read() reads from (the beginning of the file) and write() writes to (the end of the file). Seekable files, on the other hand, have no obvious place for the operations to occur. Instead, both happen at the “current” location in the file and advance the current location after the operation. When a seekable file is initially opened, the current location is at the beginning of the file, or offset 0. If 10 bytes are read, the current position is then at offset 10, and a write of 5 more bytes overwrites the data, starting with the eleventh byte in the file (which is at offset 10, where the current position was). After such a write, the current position becomes offset 15, immediately after the overwritten data.

If the current position is the end of the file and the process tries to read from the file, read() returns 0 rather than an error. If more data is written at the end of the file, the file grows just enough to accommodate the extra data and the current position becomes the new end of the file. Each file descriptor keeps track of an independent current position^[14] (it is not kept in the file’s inode), so if a file is opened multiple times by multiple processes (or by the same process, for that matter), reads and writes through one of the file descriptors do not affect the location of reads and writes made through the other file descriptor. Of course, the multiple writes could corrupt the file in other ways, so some sort of locking may be needed in these situations.

Files opened with O_APPEND have a slightly different behavior. For such files, the current position is moved to the end of the file before the kernel writes any data. After the write, the current position is moved to the end of the newly written data, as normal. For append-only files, this guarantees that the file’s current position is always at the end of the file immediately following a write().

Applications that want to read and write data from random locations in the file need to set the current position before reading and writing data, using lseek():

#include <unistd.h>

int lseek(int fd, off_t offset, int whence);

The current position for file fd is moved to offset bytes relative to whence, where whence is one of the following:

For both SEEK_CUR and SEEK_END the offset may be negative. In this case, the current position is moved toward the beginning of the file (from whence) rather than toward the end of the file. For example, the following code moves the current position to five bytes from the end of the file:

lseek(fd, -5, SEEK_END);

The lseek() system call returns the new current position in the file relative to the beginning of the file, or -1 if an error occurred. Thus, lseek(fd, 0, SEEK_END) is a simple way of finding out how large a file is, but make sure you reset the current position before trying to read from fd.

Although the current position is not disturbed by other processes that access the file at the same time,^[16] that does not mean multiple processes can safely write to a file simultaneously. Imagine the following sequence:

In this case process A would have overwritten the first five bytes of process B’s data, which is probably not what was intended. If multiple processes need to write to append to a file simultaneously, the O_APPEND flag should be used, which makes the operation atomic.

Under most POSIX systems, processes are allowed to move the current position past the end of the file. The file is grown to the appropriate size, and the current position becomes the new end of the file. The only catch is that most systems do not actually allocate any disk space for the portion of the file that was never written to; they change only the logical size of the file.

Portions of files that are “created” in this manner are known as holes. Reading from a hole in a file returns a buffer full of zeros, and writing to them could fail with an out-of-disk-space error. All of this means that lseek() should not be used to reserve disk space for later use because that space may not be allocated. If your application needs to allocate some disk space for later use, you must use write(). Files with holes in them are often used for files that have data sparsely spaced throughout them, such as files that represent hash tables.

For a simple, shell-based demonstration of file holes, look at the following example (note that /dev/zero is a character device that returns as many zeros as a process tries to read from it).

$ dd if=/dev/zero of=foo bs=1k count=10
10+0 records in
10+0 records out
$ ls -l foo
-rw-rw-r--   1 ewt      ewt       10240 Feb  6 21:50 foo
$ du foo
10 foo
$ dd if=/dev/zero of=bar bs=1k count=1 seek=9
1+0 records in
1+0 records out
$ ls -l bar
-rw-rw-r--   1 ewt      ewt       10240 Feb  6 21:50 bar
$ du bar
1       bar
$

Although both foo and bar are 10K in size, bar uses only 1K of disk space because the other 9K were seek() ed over when the file was created instead of written.

Although both read() and write() take a parameter that specifies how many bytes to read or write, neither one is guaranteed to process the requested number of bytes, even if no error has occurred. The simplest example of this is trying to read from a regular file that is already positioned at the end of the file. The system cannot actually read any bytes, but it is not exactly an error condition either. Instead, the read() call returns 0 bytes. In the same vein, if the current position was 10 bytes from the end of the file and an attempt was made to read more than 10 bytes from the file, 10 bytes would be read and the read() call would return the value 10. Again, this is not considered an error condition.

The behavior of read() also depends on whether the file was opened with O_NONBLOCK. On many file types, O_NONBLOCK does not make any difference at all. Files for which the system can guarantee an operation’s completion in a reasonable amount of time always block on reads and writes; they are sometimes referred to as fast files. This set of files includes local block devices and regular files. For other file types, such as pipes and such character devices as terminals, the process could be waiting for another process (or a human being) to either provide something for the process to read or free resources for the system to use when processing the write() request. In either case, the system has no way of knowing whether it will ever be able to complete the system call. When these files are opened with O_NONBLOCK, for each operation on the file, the system simply does as much as it is able to do immediately, and then returns to the calling process.

Nonblocking I/O is an important topic, and more examples of it are presented in Chapter 13. With the standardization of the poll() system call, however, the need for it (especially for reading) has diminished. If you find yourself using nonblocking I/O extensively, try to rethink your program in terms of poll() to see if you can make it more efficient.

To show a concrete example of reading and writing files, here is a simple reimplementation of cat. It copies stdin to stdout until there is no more input to copy.

 1: /* cat.c */
 2:
 3: #include <stdio.h>
 4: #include <unistd.h>
 5:
 6: /* While there is data on standard in (fd 0), copy it to standard
 7:    out (fd 1). Exit once no more data is available. */
 8:
 9: int main(void) {
10:     char buf[1024];
11:     int len;
12:
13:     /* len will be >= 0 while data is available, and read() is
14:        successful */
15:     while ((len = read(STDIN_FILENO, buf, sizeof(buf))) > 0) {
16:         if (write(1, buf, len) != len) {
17:             perror("write");
18:             return 1;
19:         }
20:     }
21:
22:     /* len was <= 0; If len = 0, no more data is available.
23:        Otherwise, an error occurred. */
24:     if (len < 0) {
25:         perror("read");
26:         return 1;
27:     }
28:
29:     return 0;
30: }

Although regular files automatically grow when data is written to the end of them, there is no way for the system to automatically shrink files when the data at their end is no longer needed. After all, how would the system know when data becomes extraneous? It is a process’s responsibility to notify the system when a file may be truncated at a certain point.

#include <unistd.h>

int truncate(const char * pathname, size_t length);
int ftruncate(int fd, size_t length);

The file’s size is set to length, and any data in the file past the new end of the file is lost. If length is larger than the current size of the file, the file is actually grown to the indicated length (using holes if possible), although this behavior is not guaranteed by POSIX and should not be relied on in portable programs.

When a program writes data to the file, the data is normally stored in a kernel cache until it gets written to the physical medium (such as a hard drive), but the kernel returns control to that program as soon as the data is copied into the cache. This provides major performance improvements as it allows the kernel to order writes on the disk and to group multiple writes into a single block operation. In the event of a system failure, however, it has a few drawbacks that could be important. For example, an application that assumes data is stored in a database before the index entry for that data is stored might not handle a failure that results in just the index’s getting updated.

There are a few mechanisms applications can use to wait for data to get written to the physical medium. The O_SYNC flag, discussed on page 168, causes all writes to the file to block the calling process until the medium has been updated. While this certainly works, it is not a very neat approach. Normally, applications do not need to have every operation synchronized, more often they need to make sure a set of operations has completed before beginning another set. The fsync() and fdatasync() system calls provide this semantic:

#include <unistd.h>

int fsync(int fd);
int fdatasync(int fd);

Both system calls suspend the application until all of the data for the file fd has been written. The fsync() also waits for the file’s inode information, such as the access time, to get updated.^[17] Neither of these system calls can ensure that the data gets written to nonvolatile storage, however. Modern disk drives have large caches, and a power failure could cause some data stored in those caches to get lost.

Linux’s file model does a good job of standardizing most file operations through generic functions such as read() and write() (for example, writing to a pipe is the same as writing to a file on disk). However, some devices have operations that are poorly modeled by this abstraction. For example, terminal devices, represented as character devices, need to provide a method to change the speed of the terminal, and a CD-ROM drive, represented by a block device, needs to know when it should play an audio track to help increase a programmer’s productivity.

All of these miscellaneous operations are accessed through a single system call, ioctl() (short for I/O control), which is prototyped like this:

#include <sys/ioctl.h>

int ioctl(int fd, int request, ...);

although it is almost always used like this:

int ioctl(int fd, int request, void * arg);

Whenever ioctl() is used, its first argument is the file being manipulated and the second argument specifies what operation is being requested. The final argument is usually a pointer to something, but what that something is, as well as the exact semantics of the return code, depends on what type of file fd refers to and what type of operation was requested. For some operations, arg is a long value instead of a pointer; in these instances, a typecast is normally used. There are many examples of ioctl() in this book, and you do not need to worry about using ioctl() until you come across them.

The beginning of this chapter introduced an inode as the data structure that tracks information about a file rather than just a single process’s view of it. For example, a file’s size is a constant at any given time—it does not change for different processes that have access to the file (compare this with the current position in the file, which is unique for the result of each open() rather than a property of the file itself). Linux provides three ways of reading a file’s inode information:

#include <sys/stat.h>

int stat(const char * pathname, struct stat * statbuf);
int lstat(const char *pathname, struct stat * statbuf);
int fstat(int fd, struct stat * statbuf);

The first version, stat(), returns the inode information for the file referenced by pathname, following any symlinks that are present. If you do not want to follow symlinks (to check if a file name is a symlink, for example), use lstat() instead, which does not follow them. The final version, fstat(), returns the inode referred to by an open file descriptor. All three system calls fill in the struct stat referenced by statbuf with information from the file’s inode. Table 11.3 describes the information available from struct stat.

Here is a simple program that displays information from lstat() for each file name passed as an argument. It illustrates how to use the values returned by the stat() family of functions.

 1: /* statsamp.c */
 2:
 3: /* For each file name passed on the command line, we display all of
 4:    the information lstat() returns on the file. */
 5:
 6: #include <errno.h>
 7: #include <stdio.h>
 8: #include <string.h>
 9: #include <sys/stat.h>
10: #include <sys/sysmacros.h>
11: #include <sys/types.h>
12: #include <time.h>
13: #include <unistd.h>
14:
15: #define TIME_STRING_BUF 50
16:
17: /* Make the user pass buf (of minimum length TIME_STRING_BUF) rather
18:    than using a static buf local to the function to avoid the use of
19:    static local variables and dynamic memory. No error should ever
20:    occur so we don't do any error checking. */
21: char * timeString(time_t t, char * buf) {
22:     struct tm * local;
23:
24:     local = localtime(&t);
25:     strftime(buf, TIME_STRING_BUF, "%c", local);
26:
27:     return buf;
28: }
29:
30: /* Display all of the information we get from lstat() on the file
31:     named as our sole parameter. */
32: int statFile(const char * file) {
33:     struct stat statbuf;
34:     char timeBuf[TIME_STRING_BUF];
35:
36:     if (lstat(file, &statbuf)) {
37:         fprintf(stderr, "could not lstat %s: %s
", file,
38:                 strerror(errno));
39:         return 1;
40:     }
41:
42:     printf("Filename : %s
", file);
43:     printf("On device: major %d/minor %d    Inode number: %ld
",
44:            major(statbuf.st_dev), minor(statbuf.st_dev),
45:            statbuf.st_ino);
46:     printf("Size     : %-10ld         Type: %07o       "
47:            "Permissions: %05o
", statbuf.st_size,
48:            statbuf.st_mode & S_IFMT, statbuf.st_mode & ~(S_IFMT));
49:     printf("Owner    : %d                Group: %d"
50:            "          Number of links: %d
",
51:            statbuf.st_uid, statbuf.st_gid, statbuf.st_nlink);
52:     printf("Creation time: %s
",
53:            timeString(statbuf.st_ctime, timeBuf));
54:     printf("Modified time: %s
",
55:            timeString(statbuf.st_mtime, timeBuf));
56:     printf("Access time  : %s
",
57:            timeString(statbuf.st_atime, timeBuf));
58:
59:     return 0;
60: }
61:
62: int main(int argc, const char ** argv) {
63:     int i;
64:     int rc = 0;
65:
66:     /* Call statFile() for each file name passed on the
67:        command line. */
68:     for (i = 1; i < argc; i++) {
69:         /* If statFile() ever fails, rc will end up non-zero. */
70:         rc |= statFile(argv[i]);
71:
72:         /* this prints a blank line between entries, but not after
73:           the last entry */
74:         if ((argc - i) > 1) printf("
");
75:     }
76:
77:     return rc;
78: }

Although the mode of a file provides all the information a program needs to determine whether it is allowed to access a file, testing the set of permissions is tricky and error prone. As the kernel already includes the code to validate access permissions, a simple system call is provided that lets programs determine whether they may access a file in a certain way:

#include <unistd.h>

int access(const char * pathname, int mode);

The mode is a mask that contains one or more of the following values:

access() returns 0 if the specified access modes are allowed, and an EACCES error otherwise.

A file’s access permissions and access permission modifiers are changed by the chmod() system call.

#include <sys/stat.h>

int chmod(const char * pathname, mode_t mode);
int fchmod(int fd, mode_t mode);

Although chmod() allows you to specify a path, remember that file permissions are based on the inode, not on the file name. If a file has multiple hard links to it, changing the permissions of one of the file’s names changes the file’s permissions everywhere it appears in the file system. The mode parameter may be any combination of the access and access modifier bits just discussed, bitwise OR’ed together. As there are normally quite a few of these values specified at a time, it is common for programs to specify the new permission value directly in octal. Only the root user and the owner of a file are allowed to change the access permissions for a file—all others who attempt to do so get EPERM.

Just like a file’s permissions, a file’s owner and group are stored in the file’s inode, so all hard links to a file have the same owner and group. The same system call is used to change both the owner and the group of a file.

#include <unistd.h>

int chown(const char * pathname, uid_t owner, gid_t group);
int fchown(int fd, uid_t owner, gid_t group);

The owner and group parameters specify the new owner and group for the file. If either is -1, that value is not changed. Only the root user may change the owner of a file. When a file’s owner is changed or the file is written to, the setuid bit for that file is always cleared for security reasons. Both the owner of a file and the root user may change the group that owns a file, but the owner must be a member of the group to which he is changing the file. If the file has its group execute bit set, the setgid bit is cleared for security reasons. If the setgid bit is set but the group execute bit is not, the file has mandatory locking enabled and the mode is preserved.

A file’s owner may change the mtime and atime of a file to any arbitrary value. This makes these timestamps useless as audit trails, but it allows archiving tools like tar and cpio to reset a file’s timestamps to the same values they had when the file was archived. The ctime is changed when the mtime and atime are updated, so tar and cpio are unable to restore it.

There are two ways of changing these stamps: utime() and utimes(). utime() originated in System V and was adopted by POSIX, whereas utimes() is from BSD. Both functions are equivalent; they differ only in the way the new timestamps are specified.

#include <utime.h>
int utime(const char * pathname, struct utimbuf * buf);

#include <sys/time.h>
int utimes(const char * pathname, struct timeval * tvp);

The POSIX version, utime(), takes a struct utimbuf, which is defined in <utime.h> as

struct utimbuf {
    time_t actime;
    time_t modtime;
}

BSD’s utimes() instead passes the new atime and mtime through a struct timeval, which is defined in <sys/time.h>.

struct timeval {
    long tv_sec;
    long tv_usec;
}

The tv_sec element holds the new atime; tv_usec contains the new mtime for utimes().

If NULL is passed as the second argument to either function, both timestamps are set to the current time. The new atime and mtime are specified in elapsed seconds since the epoch (just like the value time() returns), as defined in Chapter 18.

The primary file system used on Linux systems is the Third Extended File System,^[18] commonly abbreviated as ext3. Although the ext3 file system supports all the traditional features of Unix file systems, such as the meanings of the various bits in each file’s mode, it allows for other attributes for each file. Table 11.4 describes the extra attributes currently supported, along with the symbolic name for each attribute. These flags may be set and inspected through the chattr and lsattr programs.

As the ext3 extended attributes are outside the standard file system interface, they cannot be modified through chmod() like the file attributes. Instead, ioctl() is used. Recall that ioctl() is defined as

#include <sys/ioctl.h>
#include <linux/ext3_fs.h>

int ioctl(int fd, int request, void * arg);

The file whose attributes are being changed must be open, just as for fchmod(). The request is EXT3_IOC_GETFLAGS to check the current flags for the file and EXT3_IOC_SETFLAGS to set them. In either case, arg must point to an int. If EXT3_IOC_GETFLAGS is used, the long is set to the current value of the program’s flags. If EXT3_IOC_SETFLAGS is used, the new value of the file’s flags is taken from the int pointed to by arg.

The append and immutable flags may be changed only by the root user, as they restrict the operations the root user is able to perform. The other flags may be modified by either the root user or the owner of the file whose flags are being modified.

Here is a small program that displays the flags for any files specified on the command line. It works only for files on an ext3 file system,^[19] however. The ioctl() fails for files on any other type of file system.

 1: /* checkflags.c */
 2:
 3: /* For each file name passed on the command line, display
 4:    information on that file's ext3 attributes. */
 5:
 6: #include <errno.h>
 7: #include <fcntl.h>
 8: #include <linux/ext3_fs.h>
 9: #include <stdio.h>
10: #include <string.h>
11: #include <sys/ioctl.h>
12: #include <unistd.h>
13:
14: int main(int argc, const char ** argv) {
15:     const char ** filename = argv + 1;
16:     int fd;
17:     int flags;
18:
19:     /* Iterate over each file name on the command line. The last
20:        pointer in argv[] is NULL, so this while() loop is legal. */
21:     while (*filename) {
22:         /* Unlike normal attributes, ext3 attributes can only
23:            be queried if we have a file descriptor (a file name
24:            isn't sufficient). We don't need write access to query
25:            the ext3 attributes, so O_RDONLY is fine. */
26:         fd = open(*filename, O_RDONLY);
27:         if (fd < 0) {
28:             fprintf(stderr, "cannot open %s: %s
", *filename,
29:                     strerror(errno));
30:             return 1;
31:         }
32:
33:         /* This gets the attributes, and puts them into flags */
34:         if (ioctl(fd, EXT3_IOC_GETFLAGS, &flags)) {
35:             fprintf(stderr, "ioctl failed on %s: %s
", *filename,
36:                     strerror(errno));
37:             return 1;
38:         }
39:
40:         printf("%s:", *filename++);
41:
42:         /* Check for each attribute, and display a message for each
43:            one which is turned on. */
44:         if (flags & EXT3_APPEND_FL) printf(" Append");
45:         if (flags & EXT3_IMMUTABLE_FL) printf(" Immutable");
46:         if (flags & EXT3_SYNC_FL) printf(" Sync");
47:         if (flags & EXT3_NODUMP_FL) printf(" Nodump");
48:
49:         printf("
");
50:         close(fd);
51:     }
52:
53:     return 0;
54: };

The following is a similar program that sets the ext3 extended attributes for a given list of files. The first parameter must be a list of which flags should be set. Each flag is represented in the list by a single letter: A for append-only, I for immutable, S for sync, and N for the nodump flag. This program does not modify the current flags for the file; only the flags specified on the command line are set.

 1: /* setflags.c */
 2:
 3: /* The first parameter to this program is a string consisting of
 4:    0 (an empty string is okay) or more of the letters I, A, S, and
 5:    N. This string specifies which ext3 attributes should be turned
 6:    on for the files which are specified on the rest of the command
 7:    line -- the rest of the attributes are turned off. The letters
 8:    stand for immutable, append-only, sync, and nodump, respectively.
 9:
10:    For example, the command "setflags IN file1 file2" turns on the
11:    immutable and nodump flags for files file1 and file2, but turns
12:    off the sync and append-only flags for those files. */
13:
14: #include <errno.h>
15: #include <fcntl.h>
16: #include <linux/ext3_fs.h>
17: #include <stdio.h>
18: #include <string.h>
19: #include <sys/ioctl.h>
20: #include <unistd.h>
21:
22: int main(int argc, const char ** argv) {
23:     const char ** filename = argv + 1;
24:     int fd;
25:     int flags = 0;
26:
27:     /* Make sure the flags to set were specified, along with
28:        some file names. Allow a "0" to be set to indicate that all
29:        of the flags should be reset. */
30:     if (argc < 3) {
31:         fprintf(stderr, "setflags usage: [0][I][A][S][N] "
32:                         "<filenames>
");
33:         return 1;
34:     }
35:
36:     /* each letter represents a flag; set the flags which are
37:        specified */
38:     if (strchr(argv[1], 'I')) flags |= EXT3_IMMUTABLE_FL;
39:     if (strchr(argv[1], 'A')) flags |= EXT3_APPEND_FL;
40:     if (strchr(argv[1], 'S')) flags |= EXT3_SYNC_FL;
41:     if (strchr(argv[1], 'N')) flags |= EXT3_NODUMP_FL;
42:
43:     /* iterate over all of the file names in argv[] */
44:     while (*(++filename)) {
45:         /* Unlike normal attributes, ext3 attributes can only
46:            be set if we have a file descriptor (a file name
47:            isn't sufficient). We don't need write access to set
48:            the ext3 attributes, so O_RDONLY is fine. */
49:         fd = open(*filename, O_RDONLY);
50:         if (fd < 0) {
51:             fprintf(stderr, "cannot open %s: %s
", *filename,
52:                     strerror(errno));
53:             return 1;
54:         }
55:
56:         /* Sets the attributes as specified by the contents of
57:            flags. */
58:         if (ioctl(fd, EXT3_IOC_SETFLAGS, &flags)) {
59:             fprintf(stderr, "ioctl failed on %s: %s
", *filename,
60:                     strerror(errno));
61:             return 1;
62:         }
63:         close(fd);
64:     }
65:
66:     return 0;
67: };

Processes create device file entries and named pipes in the file system through mknod().

#include <fcntl.h>
#include <unistd.h>

int mknod(const char * pathname, mode_t mode, dev_t dev);

The pathname is the file name to create, mode is both the access mode of the new file (which gets modified by the current umask) and the new file type (S_IFIFO, S_IFBLK, or S_IFCHR). The final parameter, dev, contains the major and minor numbers of the device to create. The type of device (character or block) and the major number of the device tell the kernel which device driver is responsible for operations on that device file. The minor number is used internally by the device driver to differentiate among multiple devices it provides. Only the root user is allowed to create device files; all users may create named pipes.

The <sys/sysmacros.h> header file provides three macros for manipulating dev_t values. The makedev() macro takes a major number as its first argument and a minor number as its second, and it returns a dev_t suitable for mknod(). The major() and minor() macros take a dev_t value as their sole argument and return the device’s major and minor numbers, respectively.

The mknod program available under Linux provides a user-level interface to the mknod() system call (see man 1 mknod for details). Here is a simple reimplementation of mknod to illustrate the mknod() system call. Notice that the program creates the file with mode 0666 (giving read and write access to all users), and it depends on the process’s umask setting to get the permissions right.

 1: /* mknod.c */
 2:
 3: /* Create the device or named pipe specified on the command line.
 4:    See the mknod(1) man page for details on the command line
 5:    parameters. */
 6:
 7: #include <errno.h>
 8: #include <stdio.h>
 9: #include <stdlib.h>
10: #include <string.h>
11: #include <sys/stat.h>
12: #include <sys/sysmacros.h>
13: #include <unistd.h>
14:
15: void usage(void) {
16:      fprintf(stderr, "usage: mknod <path> [b|c|u|p] "
17:                      "<major> <minor>
");
18:      exit(1);
19: }
20:
21: int main(int argc, const char ** argv) {
22:     int major = 0, minor = 0;
23:     const char * path;
24:     int mode = 0666;
25:     char *end;
26:     int args;
27:
28:     /* We always need at least the type of inode to create, and
29:        the path for it. */
30:     if (argc < 3) usage();
31:
32:     path = argv[1];
33:
34:     /* the second argument tells us the type of node to create */
35:     if (!strcmp(argv[2], "b")) {
36:         mode |= S_IFBLK;
37:         args = 5;
38:     } else if (!strcmp(argv[2], "c") || !strcmp(argv[2], "u")) {
39:         mode |= S_IFCHR;
40:         args = 5;
41:     } else if (!strcmp(argv[2], "p")) {
42:         mode |= S_IFIFO;
43:         args = 3;
44:     } else {
45:         fprintf(stderr, "unknown node type %s
", argv[2]);
46:         return 1;
47:     }
48:
49:     /* args tells us how many parameters to expect, as we need more
50:        information to create device files than named pipes */
51:     if (argc != args) usage();
52:
53:     if (args == 5) {
54:         /* get the major and minor numbers for the device file to
55:            create */
56:         major = strtol(argv[3], &end, 0);
57:         if (*end) {
58:             fprintf(stderr, "bad major number %s
", argv[3]);
59:             return 1;
60:         }
61:
62:         minor = strtol(argv[4], &end, 0);
63:         if (*end) {
64:             fprintf(stderr, "bad minor number %s
", argv[4]);
65:             return 1;
66:         }
67:     }
68:
69:     /* if we're creating a named pipe, the final parameter is
70:        ignored */
71:     if (mknod(path, mode, makedev(major, minor))) {
72:         fprintf(stderr, "mknod failed: %s
", strerror(errno));
73:         return 1;
74:     }
75:
76:     return 0;
77: }

When multiple file names in the file system refer to a single inode, the files are called hard links to one other. All the file names must reside in the same physical file system (normally, this means they must all be on the same device). When a file has multiple hard links, each of those file names is an exact peer—there is no way to tell which file name was originally used. One benefit of this model is that removing one hard link does not remove the file from the device—it stays until all the links to it are removed. The link() system call links a new file name to an existing inode.

#include <unistd.h>

int link(const char * origpath, const char * newpath);

The origpath refers to a pathname that already exists; newpath is the path for the new hard link. Any user may create a link to a file he has read access to, as long as he has write access for the directory in which he is creating the link and execute permissions on the directory origpath is in. Only the root user has ever been allowed to create hard links to directories, but doing so was generally a bad idea, because most file systems and several utilities do not handle it very well; it is now entirely disallowed.

Symbolic links are a more flexible type of link than hard links, but they do not share the peer relationship that hard links enjoy. Whereas hard links share an inode, symbolic links point to other file names. If the destination file name is removed, the symbolic link then points to a file that does not exist, resulting in a dangling link. Using symbolic links between subdirectories is common, and symbolic links may also cross physical file system boundaries, which hard links cannot.

Almost all system calls that access files by their pathname automatically follow symbolic links to find the proper inode. The following calls do not follow symbolic links under Linux:

Creating symbolic links is just like creating hard links, but the symlink() system call is used instead.

#include <unistd.h>

int symlink(const char * origpath, const char * newpath);

If the call is successful, the file newpath gets created as a symbolic link pointing to origpath (often newpath is said to contain oldpath as its value).

Finding the value of the symbolic link is a bit more complicated.

#include <unistd.h>

int readlink(const char * pathname, char * buf,
             size_t bufsiz);

The buffer that buf points to is filled in by the contents of the pathname symlink as long as buf is long enough to hold the contents. bufsize should contain the length of buf in bytes. Usually, the PATH_MAX constant is used for the size of the buffer, as that should be big enough to hold the contents of any symlink.^[20] One oddity of readlink() is that it does not end the string it writes to buf with a '' character, so buf is not a valid C string even if readlink() succeeds. It instead returns the number of bytes written to buf on success, and -1 on error. Thanks to this quirk, code that uses readlink() often looks like this:

char buf[PATH_MAX + 1];
int bytes;

if ((bytes = readlink(pathname, buf, sizeof(buf) - 1)) < 0) {
    perror("error in readlink");
} else {
    buf[bytes] = '';
}

Removing a file removes the pointer to its inode and removes the file’s data if there are no other hard links to the file. If any processes have the file open, the file’s inode is preserved until the final process closes the file, and then the inode and the file’s data are both discarded. As there is no way of forcing a file to be removed immediately, this operation is called unlinking the file, since it removes a file name/inode link.

#include <unistd.h>

int unlink(char * pathname);

A file name may be changed to any other file name as long as both names are on the same physical partition (this is the same limit that applies to creating hard links). If the new file name already references a file, that name is unlinked before the move takes place. The rename() system call is guaranteed to be atomic. Other processes on the system always see the file in existence under one name or the other; there is no point at which the file does not exist under either name, nor under both names. As open files are not related to file names (only to inodes), renaming a file that other processes have open does not affect those other processes in any way. Here is what the system call looks like:

#include <unistd.h>

int rename(const char * oldpath, const char * newpath);

After the call, the file referenced by oldpath may be referenced by newpath, but not by oldpath.

The append mode (as indicated by the O_APPEND flag when the file is opened) and nonblocking mode (the O_NONBLOCK flag), can be turned on and off after a file has already been opened through the fcntl()’s F_SETFL command. The arg parameter should be the flags that should be set—if one of the flags is not specified, it is turned off for fd.

F_GETFL may be used to query the current flags for the file. It returns all the flags, including the read/write mode the file was opened in. F_SETFL allows only setting the flags mentioned above; any other flags that are present in the arg parameter are ignored.

fcntl(fd, F_SETFL, fcntl(fd, F_GETFL, 0) | O_RDONLY);

is perfectly legal, but it does not accomplish anything. Turning on append mode for a file descriptor looks like this:

fcntl(fd, F_SETFL, fcntl(fd, F_GETFL, 0) | O_APPEND);

Note that care was taken to preserve the O_NONBLOCK setting. Turning append mode off looks similar:

fcntl(fd, F_SETFL, fcntl(fd, F_GETFL, 0) & ~O_APPEND);

During an exec() system call, file descriptors are normally left open for the new program to use. In certain cases, you may wish to have files closed when you call exec(). Rather than closing them by hand, you can ask the system to close a certain file descriptor when exec() is called through fcntl()’s F_GETFD and F_SETFD commands. If the close-on-exec flag is set when F_GETFD is used, fcntl() returns non-0; otherwise, it returns 0. The close-on-exec flag is set through F_SETFD; it is disabled if arg is 0 and enabled otherwise.

Here is how you would force fd to be closed when the process exec() s:

fcntl(fd, F_SETFD, 1);

Occasionally, a process needs to create a new file descriptor that references a file that is already open. Shells use this functionality to redirect standard input, output, and error at the user’s request. If the process does not care what file descriptor is used for the new reference, it should use dup().

#include <unistd.h>

int dup(int oldfd);

dup() returns a file descriptor that references the same inode as oldfd, or-1 on an error. The oldfd is still a valid file descriptor and still references the original file. The new file descriptor is always the smallest file descriptor currently available. If the process needs the new file descriptor to have a particular value (such as 0 to change its standard input), it should use dup2() instead.

#include <unistd.h>

int dup2(int oldfd, int newfd);

If newfd references an already open file descriptor, that file descriptor is closed. If the call succeeds, it returns the new file descriptor and newfd references the same file as oldfd. The fcntl() system call provides almost the same functionality through the F_DUPFD command. The first argument, fd, is the already open file descriptor. The new file descriptor is the first available file descriptor that is the same or larger than the last argument to fcntl(). (This is different than how dup2() works.) You could implement dup2() through fcntl() like this:

int dup2(int oldfd, int newfd) {
    close(newfd);                   /* ensure newfd is available */
    return fcntl(oldfd, F_DUPFD, newfd);
}

Creating two duped file descriptors that reference the same file is not the same as opening a file twice. Nearly all duped file descriptors’ attributes are shared; they share a common current position, access mode, and locks. (These items are stored in a file structure,^[21] one of which is created each time the file is opened. A file descriptor refers to a file structure, and dup() ed file descriptors refer to a single file structure.) The only attribute that can be independently controlled for the two file descriptors is their close-on-exec status. After the process has fork() ed, the parent’s open files are inherited by the child, and those pairs of file descriptors (one in the parent and one in the new child) behave exactly like a file descriptor that has been duped with the current position and most other attributes being shared.^[22]

Although ladsh1.c included the concept of a job as multiple processes (presumably tied together by pipes), it did not provide a way of specifying which files to use for a child’s input and output. To allow for this, new data structures are introduced and existing ones modified.

24:                        REDIRECT_APPEND };
25:
26: struct redirectionSpecifier {
27:     enum redirectionType type;  /* type of redirection */
28:     int fd;                 /* file descriptor being redirected */
29:     char * filename;        /* file to redirect fd to */
30: };
31:
32: struct childProgram {
33:     pid_t pid;              /* 0 if exited */
34:     char ** argv;           /* program name and arguments */
35:     int numRedirections;    /* elements in redirection array */
36:     struct redirectionSpecifier * redirections; /* I/O redirs */
37: };

struct redirectionSpecifier tells ladsh2.c how to set up a single file descriptor. It contains an enum redirectionType that tells us whether this redirection is an input redirection, an output redirection that should be appended to an already existing file, or an output redirection that replaces any existing file. It also includes the file descriptor that is being redirected, as well as the name of the file involved. Each child program (struct childProgram) now specifies an arbitrary number of redirections for itself.

These new data structures are not involved in setting up pipes between processes. As a job is defined as multiple child processes with pipes tying them together, there is no need for more explicit information describing the pipes. Figure 11.1 shows how these new data structures would look for the command tail < input-file | sort > output-file.

Figure 11.1. Job Data Structures for ladsh2.c

Once parseCommand() has set up the data structures properly, running the commands in the proper sequence is easy enough, as long as you watch the details. First of all, we added a loop to runCommand() to start the child processes, because there could now be multiple children. Before entering the loop, we set up nextin and nextout, which are the file descriptors to use for the standard input and the standard output of the next process we start. To begin with, we use the same stdin and stdout as the shell.

Now we take a look at what happens inside the loop. The basic idea is as follows:

Here is how these ideas translate into C:

365:     nextin = 0, nextout = 1;
366:     for (i = 0; i < newJob.numProgs; i++) {
367:         if ((i + 1) < newJob.numProgs) {
368:             pipe(pipefds);
369:             nextout = pipefds[1];
370:         } else {
371:             nextout = 1;
372:         }
373:
374:         if (!(newJob.progs[i].pid = fork())) {
375:             if (nextin != 0) {
376:                 dup2(nextin, 0);
377:                 close(nextin);
378:             }
379:
380:             if (nextout != 1) {
381:                 dup2(nextout, 1);
382:                 close(nextout);
383:             }
384:
385:             /* explicit redirections override pipes */
386:             setupRedirections(newJob.progs + i);
387:
388:             execvp(newJob.progs[i].argv[0], newJob.progs[i].argv);
389:             fprintf(stderr, "exec() of %s failed: %s
",
390:                     newJob.progs[i].argv[0],
391:                     strerror(errno));
392:             exit(1);
393:         }
394:
395:         /* put our child in the process group whose leader is the
396:            first process in this pipe */
397:         setpgid(newJob.progs[i].pid, newJob.progs[0].pid);
398:
399:         if (nextin != 0) close(nextin);
400:         if (nextout != 1) close(nextout);
401:
402:         /* If there isn't another process, nextin is garbage
403:            but it doesn't matter */
404:         nextin = pipefds[0];

The only other code added to ladsh2.c to allow redirection was setupRedirections(), the source of which appears unchanged in all subsequent versions of ladsh. Its job is to process the struct redirectionSpecifier specifiers for a child job and modify the child’s file descriptors as appropriate. We recommend reading over the function as it appears in Appendix B to ensure you understand its implementation.