The 2001 POSIX standard defines a large number of possible values for errno. Many of these are related to networking, IPC, or other specialized tasks. The manpage for each system call describes the possible errno values that can occur; thus, you can write code to check for particular errors and handle them specially if need be. The possible values are defined by symbolic constants. Table 4.1 lists the constants provided by GLIBC.

Many systems provide other error values as well, and older systems may not have all the errors just listed. You should check your local intro(2) and errno(2) manpages for the full story.

Initially, we use errno only for error reporting. There are two useful functions for error reporting. The first is perror():

#include <stdio.h>                                        ISO C

void perror(const char *s);

The perror() function prints a program-supplied string, followed by a colon, and then a string describing the value of errno:

if (some_system_call(param1, param2) < 0) {
    perror("system call failed");
    return 1;
}

We prefer the strerror() function, which takes an error value parameter and returns a pointer to a string describing the error:

#include <string.h>                                       ISO C

char *strerror(int errnum);

strerror() provides maximum flexibility in error reporting, since fprintf() makes it possible to print the error in any way we like:

if (some_system_call(param1, param2) < 0) {
    fprintf(stderr, "%s: %d, %d: some_system_call failed: %s
",
            argv[0], param1, param2, strerror(errno));
    return 1;
}

You will see many examples of both functions throughout the book.

C provides several special macros for use in error reporting. The most widely used are __FILE__ and __LINE__, which expand to the name of the source file and the current line number in that file. These have been available in C since its beginning. C99 defines an additional predefined identifier, __func__, which represents the name of the current function as a character string. The macros are used like this:

if (some_system_call(param1, param2) < 0) {
    fprintf(stderr, "%s: %s (%s %d): some_system_call(%d, %d) failed: %s
",
        argv[0], __func__, __FILE__, __LINE__,
 param1, param2, strerror(errno));
    return 1;
}

Here, the error message includes not only the program’s name but also the function name, source file name, and line number. The full list of identifiers useful for diagnostics is provided in Table 4.2.

The use of __FILE__ and __LINE__ was quite popular in the early days of Unix, when most people had source code and could find the error and fix it. As Unix systems became more commercial, use of these identifiers gradually diminished, since knowing the source code location isn’t of much help to someone who only has a binary executable.

Today, although GNU/Linux systems come with source code, said source code often isn’t installed by default. Thus, using these identifiers for error messages doesn’t seem to provide much additional value. The GNU Coding Standards don’t even mention them.

A file descriptor is an integer value. Valid file descriptors start at 0 and go up to some system-defined limit. These integers are in fact simple indexes into each process’s table of open files. (This table is maintained inside the operating system; it is not accessible to a running program.) On most modern systems, the size of the table is large. The command ’ulimit -n’ prints the value:

$ ulimit -n
1024

From C, the maximum number of open files is returned by the getdtablesize() (get descriptor table size) function:

#include <unistd.h>                                              Common

int getdtablesize(void);

This small program prints the result of the function:

/* ch04-maxfds.c --- Demonstrate getdtablesize(). */

#include <stdio.h>         /* for fprintf(), stderr, BUFSIZ */
#include <unistd.h>        /* for ssize_t */

int
main(int argc, char **argv)
{
   printf("max fds: %d
", getdtablesize());
   exit(0);
}

When compiled and run, not surprisingly the program prints the same value as printed by ulimit:

$ ch04-maxfds
max fds: 1024

File descriptors are held in normal int variables; it is typical to see declarations of the form ’int fd’ for use with I/O system calls. There is no predefined type for file descriptors.

In the usual case, every program starts running with three file descriptors already opened for it. These are standard input, standard output, and standard error, on file descriptors 0, 1, and 2, respectively. (If not otherwise redirected, each one is connected to your keyboard and screen.)

int fd = 0;

enum { Stdin, Stdout, Stderr };

New file descriptors are obtained (among other sources) from the open() system call. This system call opens a file for reading or writing and returns a new file descriptor for subsequent operations on the file. We saw the declaration earlier:

#include <sys/types.h>                                       POSIX
#include <sys/stat.h>
#include <fcntl.h>
#include <unistd.h>

int open(const char *pathname, int flags, mode_t mode);

The three arguments are as follows:

const char *pathname

int flags

mode_t mode

The return value from open() is either the new file descriptor or -1 to indicate an error, in which case errno will be set. For simple I/O, the flags argument should be one of the values in Table 4.3.

We will see example code shortly. Additional values for flags are described in Section 4.6, “Creating Files,” page 106. Much early Unix code didn’t use the symbolic values. Instead, the numeric value was used. Today this is considered bad practice, but we present the values so that you’ll recognize their meanings if you see them.

The close() system call closes a file: The entry for it in the system’s file descriptor table is marked as unused, and no further operations may be done with that file descriptor. The declaration is

#include <unistd.h>                                              POSIX

int close(int fd);

The return value is 0 on success, -1 on error. There isn’t much you can do if an error does occur, other than report it. Errors closing files are unusual, but not unheard of, particularly for files being accessed over a network. Thus, it’s good practice to check the return value, particularly for files opened for writing.

If you choose to ignore the return value, specifically cast it to void, to signify that you don’t care about the result:

(void) close(fd);      /* throw away return value */

The flip side of this advice is that too many casts to void tend to the clutter the code. For example, despite the “always check the return value” principle, it’s exceedingly rare to see code that checks the return value of printf() or bothers to cast it to void. As with many aspects of C programming, experience and judgment should be applied here too.

As mentioned, the number of open files, while large, is limited, and you should always close files when you’re done with them. If you don’t, you will eventually run out of file descriptors, a situation that leads to a lack of robustness on the part of your program.

The system closes all open files when a process exits, but—except for 0, 1, and 2—it’s bad form to rely on this.

When open() returns a new file descriptor, it always returns the lowest unused integer value. Always. Thus, if file descriptors 0–6 are open and the program closes file descriptor 5, then the next call to open() returns 5, not 7. This behavior is important; we see later in the book how it’s used to cleanly implement many important Unix features, such as I/O redirection and piping.

#include <stdio.h>                                              POSIX

int fileno(FILE *stream);

int i;

/* leave 0, 1, and 2 alone */
for (i = 3; i < getdtablesize(); i++)
    (void) close(i);

int i, fds;

for (i = 3, fds = getdtablesize(); i < fds; i++)
    (void) close(i);

I/O is accomplished with the read() and write() system calls, respectively:

#include <sys/types.h>                                         POSIX
#include <sys/stat.h>
#include <fcntl.h>
#include <unistd.h>

ssize_t read(int fd, void *buf, size_t count);
ssize_t write(int fd, const void *buf, size_t count);

Each function is about as simple as can be. The arguments are the file descriptor for the open file, a pointer to a buffer to read data into or to write data from, and the number of bytes to read or write.

The return value is the number of bytes actually read or written. (This number can be smaller than the requested amount: For a read operation this happens when fewer than count bytes are left in the file, and for a write operation it happens if a disk fills up or some other error occurs.) The return value is -1 if an error occurred, in which case errno indicates the error. When read() returns 0, it means that end-of-file has been reached.

We can now show the rest of the code for ch04-cat. The process() routine uses 0 if the input filename is "-", for standard input (lines 50 and 51). Otherwise, it opens the given file:

36  /*
37   * process --- do something with the file, in this case,
38   *             send it to stdout (fd 1).
39   *             Returns 0 if all OK, 1 otherwise.
40   */
41
42  int
43  process(char *file)
44  {
45      int fd;
46      ssize_t rcount, wcount;
47      char buffer[BUFSIZ];
48      int errors = 0;
49
50      if (strcmp(file, "-") == 0)
51          fd = 0;
52      else if ((fd = open(file, O_RDONLY)) < 0) {
53          fprintf(stderr, "%s: %s: cannot open for reading: %s
",
54                  myname, file, strerror(errno));
55          return 1;
56      }

The buffer buffer (line 47) is of size BUFSIZ; this constant is defined by <stdio.h> to be the “optimal” block size for I/O. Although the value for BUFSIZ varies across systems, code that uses this constant is clean and portable.

The core of the routine is the following loop, which repeatedly reads data until either end-of-file or an error is encountered:

58      while ((rcount = read(fd, buffer, sizeof buffer)) > 0) {
59          wcount = write(1, buffer, rcount);
60          if (wcount != rcount) {
61              fprintf(stderr, "%s: %s: write error: %s
",
62                      myname, file, strerror(errno));
63              errors++;
64              break;
65          }
66      }

The rcount and wcount variables (line 45) are of type ssize_t, “signed size_t,” which allows them to hold negative values. Note that the count value passed to write() is the return value from read() (line 59). While we want to read fixed-size BUFSIZ chunks, it is unlikely that the file itself is a multiple of BUFSIZ bytes big. When the final, smaller, chunk of bytes is read from the file, the return value indicates how many bytes of buffer received new data. Only those bytes should be copied to standard output, not the entire buffer.

The test ’wcount != rcount’ on line 60 is the correct way to check for write errors; if some, but not all, of the data were written, then wcount will be positive but smaller than rcount.

Finally, process() checks for read errors (lines 68–72) and then attempts to close the file. In the (unlikely) event that close() fails (line 75), it prints an error message. Avoiding the close of standard input isn’t strictly necessary in this program, but it’s a good habit to develop for writing larger programs, in case other code elsewhere wants to do something with it or if a child program will inherit it. The last statement (line 82) returns 1 if there were errors, 0 otherwise.

68      if (rcount < 0) {
69          fprintf(stderr, "%s: %s: read error: %s
",
70                  myname, file, strerror(errno));
71          errors++;
72      }
73
74      if (fd != 0) {
75          if (close(fd) < 0) {
76              fprintf(stderr, "%s: %s: close error: %s
",
77                  myname, file, strerror(errno));
78              errors++;
79          }
80      }
81
82      return (errors != 0);
83  }

ch04-cat checks every system call for errors. While this is tedious, it provides robustness (or at least clarity): When something goes wrong, ch04-cat prints an error message that is as specific as possible. The combination of errno and strerror() makes this easy to do. That’s it for ch04-cat, only 88 lines of code!

To sum up, there are several points to understand about Unix I/O:

I/O is uninterpreted.

I/O is flexible.

I/O is simple.

I/O can be partial.

As promised, here is the V7 version of cat.^[2] It begins by checking for options. The V7 cat accepts a single option, -u, for doing unbuffered output.

The basic design is similar to the one shown above; it loops over the files named by the command-line arguments and reads each file, one character at a time, sending the characters to standard output. Unlike our version, it uses the <stdio.h> facilities. In many ways code using the Standard I/O library is easier to read and write, since all buffering issues are hidden by the library.

 1  /*
 2   * Concatenate files.
 3   */
 4
 5  #include <stdio.h>
 6  #include <sys/types.h>
 7  #include <sys/stat.h>
 8
 9  char    stdbuf[BUFSIZ];
10
11  main(argc, argv)                        int main(int argc, char **argv)
12  char **argv;
13  {
14      int fflg = 0;
15      register FILE *fi;
16      register c;
17      int dev, ino = -1;
18      struct stat statb;
19
20      setbuf(stdout, stdbuf);
21      for( ; argc>1 && argv[1][0]=='-'; argc--,argv++) {
22          switch(argv[1] [1]) {             Process options
23          case 0:
24              break;
25          case 'u':
26              setbuf(stdout, (char *)NULL);
27              continue;
28          }
29          break;
30      }
31      fstat(fileno(stdout), &statb);         Lines 31–36 explained in Chapter 5
32      statb.st_mode &= S_IFMT;
33      if (statb.st_mode!=S_IFCHR && statb.st_mode!=S_IFBLK) {
34          dev = statb.st_dev;
35          ino = statb.st_ino;
36      }
37      if (argc < 2) {
38          argc = 2;
39          fflg++;
40      }
41      while (--argc > 0) {                   Loop over files
42          if (fflg || (*++argv) [0]=='-' && (*argv) [1]=='')
43              fi = stdin;
44          else {
45              if ((fi = fopen(*argv, "r")) == NULL) {
46                  fprintf(stderr, "cat: can't open %s
", *argv);
47                  continue;
48              }
49          }
50          fstat(fileno(fi), &statb);         Lines 50–56 explained in Chapter 5
51          if (statb.st_dev==dev && statb.st_ino==ino) {
52              fprintf(stderr, "cat: input %s is output
",
53                 fflg?"-": *argv);
54              fclose(fi);
55              continue;
56          }
57          while ((c = getc(fi)) != EOF)      Copy file contents to stdout
58              putchar(c);
59          if (fi!=stdin)
60              fclose(fi);
61      }
62      return(0);
63  }

Of note is that the program always exits successfully (line 62); it could have been written to note errors and indicate them in main()’s return value. (The mechanics of process exiting and the meaning of different exit status values are discussed in Section 9.1.5.1, “Defining Process Exit Status,” page 300.)

The code dealing with the struct stat and the fstat() function (lines 31–36 and 50–56) is undoubtedly opaque, since we haven’t yet covered these functions, and won’t until the next chapter. (But do note the use of fileno() on line 50 to get at the underlying file descriptor associated with the FILE * variables.) The idea behind the code is to make sure that no input file is the same as the output file. This is intended to prevent infinite file growth, in case of a command like this:

$ cat myfile >> myfile                       Append one copy of myfile onto itself?

And indeed, the check works:

$ echo hi > myfile                           Create a file
$ v7cat myfile >> myfile                     Attempt to append it onto itself
cat: input myfile is output

If you try this with ch04-cat, it will keep running, and myfile will keep growing until you interrupt it. The GNU version of cat does perform the check. Note that something like the following is beyond cat’s control:

$ v7cat < myfile > myfile
cat: input - is output
$ ls -l myfile
-rw-r--r-- 1 arnold devel               0 Mar 24 14:17 myfile

In this case, it’s too late because the shell truncated myfile (with the > operator) before cat ever gets a chance to examine the file!

In Section 5.4.4.2, “The V7 cat Revisited,” page 150, we explain the struct stat code.

As a GNU/Linux user, you are familiar with file permissions as printed by ’ls -l’: read, write, and execute for each of user (the file’s owner), group, and other. The various combinations are often expressed in octal, particularly for the chmod and umask commands. For example, file permissions -rw-r--r-- is equivalent to octal 0644 and -rwxr-xr-x is equivalent to octal 0755. (The leading 0 is C’s notation for octal values.)

When you create a file, you must know the protections to be given to the new file. You can do this as a raw octal number if you choose, and indeed it’s not uncommon to see such numbers in older code. However, it is better to use a bitwise OR of one or more of the symbolic constants from <sys/stat.h>, described in Table 4.5.

The following fragment shows how to create variables representing permissions -rw-r--r-- and -rwxr-xr-x (0644 and 0755 respectively):

mode_t rw_mode, rwx_mode;

rw_mode  = S_IRUSR | S_IWUSR | S_IRGRP | S_IROTH;               /* 0644 */
rwx_mode = S_IRWXU | S_IRGRP | S_IXGRP | S_IROTH | S_IXOTH;     /* 0755 */

Older code used S_IREAD, S_IWRITE, and S_IEXEC together with bit shifting to produce the same results:

mode_t rw_mode, rwx_mode;

rw_mode  = (S_IREAD|S_IWRITE) | (S_IREAD >> 3) | (S_IREAD >> 6); /* 0644 */
rwx_mode = (S_IREAD|S_IWRITE|S_IEXEC) |
           ((S_IREAD|S_IEXEC) >> 3) | ((S_IREAD|S_IEXEC) >> 6);  /* 0755 */

Unfortunately, neither notation is incredibly clear. The modern version is preferred since each permission bit has its own name and there is less opportunity to do the bitwise operations incorrectly.

The additional permission bits shown in Table 4.6 are available for use when you are changing a file’s permission, but they should not be used when you initially create a file. Whether these bits may be included varies wildly by operating system. It’s best not to try; rather, you should explicitly change the permissions after the file is created. (Changing permission is described in Section 5.5.2, “Changing Permissions: chmod() and fchmod(),” page 156. The meanings of these bits is discussed in Chapter 11, “Permissions and User and Group ID Numbers,” page 403.)

When standard utilities create files, the default permissions they use are -rw-rw-rw- (or 0666). Because most users prefer to avoid having files that are world-writable, each process carries with it a umask. The umask is a set of permission bits indicating those bits that should never be allowed when new files are created. (The umask is not used when changing permissions.) Conceptually, the operation that occurs is

actual_permissions = (requested_permissions & (~umask));

The umask is usually set by the umask command in $HOME/.profile when you log in. From a C program, it’s set with the umask() system call:

#include <sys/types.h>                                      POSIX
#include <sys/stat.h>

mode_t umask(mode_t mask);

The return value is the old umask. Thus, to determine the current mask, you must set it to a value and then reset it (or change it, as desired):

mode_t mask = umask(0);       /* retrieve current mask */
(void) umask(mask);           /* restore it */

Here is an example of the umask in action, at the shell level:

$ umask                                   Show the current mask
0022
$ touch newfile                           Create a file
$ ls -l newfile                           Show permissions of new file
-rw-r--r--    1 arnold   devel         0 Mar 24 15:43 newfile
$ umask 0                                 Set mask to empty
$ touch newfile2                          Create a second file
$ ls -l newfile2                          Show permissions of new file
-rw-rw-rw-    1 arnold   devel         0 Mar 24 15:44 newfile2

The creat()^[4] system call creates new files. It is declared as follows:

#include <sys/types.h>                                    POSIX
#include <sys/stat.h>
#include <fcntl.h>

int creat(const char *pathname, mode_t mode);

The mode argument represents the permissions for the new file (as discussed in the previous section). The file named by pathname is created, with the given permission as modified by the umask. It is opened for writing (only), and the return value is the file descriptor for the new file or -1 if there was a problem. In this case, errno indicates the error. If the file already exists, it will be truncated when opened.

In all other respects, file descriptors returned by creat() are the same as those returned by open(); they’re used for writing and seeking and must be closed with close():

int fd, count;

/* Error checking omitted for brevity */
fd = creat("/some/new/file", 0666);
count = write(fd, "some data
", 10);
(void) close(fd);

You may recall the declaration for open():

int open(const char *pathname, int flags, mode_t mode);

Earlier, we said that when opening a file for plain I/O, we could ignore the mode argument. Having seen creat(), though, you can probably guess that open() can also be used for creating files and that the mode argument is used in this case. This is indeed true.

Besides the O_RDONLY, O_WRONLY, and O_RDWR flags, additional flags may be bitwise OR’d when open() is called. The POSIX standard mandates a number of these additional flags. Table 4.7 presents the flags that are used for most mundane applications.

Given O_APPEND and O_TRUNC, you can imagine how the shell might open or create files corresponding to the > and >> operators. For example:

int fd;
extern char *filename;
mode_t mode = S_IRUSR|S_IWUSR|S_IRGRP|S_IWGRP|S_IROTH|S_IWOTH;  /* 0666 */

fd = open(filename, O_CREAT|O_WRONLY|O_TRUNC, mode);            /* for > */

fd = open(filename, O_CREAT|O_WRONLY|O_APPEND, mode);           /* for >> */

Note that the O_EXCL flag would not be used here, since for both > and >>, it’s not an error for the file to exist. Remember also that the system applies the umask to the requested permissions.

Also, it’s easy to see that, at least conceptually, creat() could be written this easily:

int creat(const char *path, mode_t mode)
{
    return open(path, O_CREAT|O_WRONLY|O_TRUNC, mode);
}

Modern systems provide additional flags whose uses are more specialized. Table 4.8 describes them briefly.

The O_DSYNC, O_RSYNC, and O_SYNC flags need some explanation. Unix systems (including Linux) maintain an internal cache of disk blocks, called the buffer cache. When the write() system call returns, the data passed to the operating system have been copied to a buffer in the buffer cache. They are not necessarily written out to the disk.

The buffer cache provides considerable performance improvement: Since disk I/O is often an order of magnitude or more slower than CPU and memory operations, programs would slow down considerably if they had to wait for every write to go all the way through to the disk. In addition, if data have recently been written to a file, a subsequent read of that same data will find the information already in the buffer cache, where it can be returned immediately instead of having to wait for an I/O operation to read it from the disk.

Unix systems also do read-ahead; since most reads are sequential, upon reading one block, the operating system will read several more consecutive disk blocks so that their information will already be in the buffer cache when a program asks for it. If multiple programs are reading the same file, they all benefit since they will all get their data from the same copy of the file’s disk blocks in the buffer cache.

All of this caching is wonderful, but of course there’s no free lunch. While data are in the buffer cache and before they have been written to disk, there’s a small—but very real—window in which disaster can strike; for example, if the power goes out. Modern disk drives exacerbate this problem: Many have their own internal buffers, so while data may have made it to the drive, it may not have made it onto the media when the power goes out! This can be a significant issue for small systems that aren’t in a data center with controlled power or that don’t have an uninterruptible power supply (UPS).^[5]

For most applications, the chance that data in the buffer cache might be inadvertently lost is acceptably small. However, for some applications, any such chance is not acceptable. Thus, the notion of synchronous I/O was added to Unix systems, whereby a program can be guaranteed that if a system call has returned, the data are safely written on a physical storage device.

The O_DSYNC flag guarantees data integrity; the data and any other information that the operating system needs to find the data are written to disk before write() returns. However, metadata, such as access and modification times, may not be written to disk. The O_SYNC flag requires that metadata also be written to disk before write() returns. (Here too there is no free lunch; synchronous writes can seriously affect the performance of a program, slowing it down noticeably.)

The O_RSYNC flag is for data reads: If read() finds data in the buffer cache that were scheduled for writing to disk, then read() won’t return that data until they have been written to disk. The other two flags can affect this: In particular, O_SYNC will cause read() to wait until the file metadata have been written out as well.