Non-Blocking I/O

A process is put to sleep when performing I/O for one or more of the following reasons:

• A read request must wait for input data to arrive.

• A write request must wait until previously written data has been written to the media.

• A device must be opened, such as a modem terminal line waiting for a carrier or a FIFO waiting for a reader.

• Mandatory locking is enabled on files, causing a wait for locking on a read or a write system call.

Conceptually, the simplest solution to this problem is to not put the process to sleep. When the I/O cannot be completed, the system call returns an error indicating that it cannot succeed at this time. This is non-blocking I/O.

Opening Files in Non-Blocking Mode

One method of specifying to the UNIX kernel that you want to use non-blocking I/O is to open with the O_NONBLOCK flag:

#include <fcntl.h>
int open(const char *path, int flags, ...);

where the flags argument is set to include O_NONBLOCK, to open in non-blocking mode.

The O_NONBLOCK flag prevents the open(2) call from suspending the execution of the calling process if it must wait for some reason. This can happen, for example, when opening a terminal line that must have a modem carrier. With the O_NONBLOCK flag provided, the open call returns success immediately.

Subsequently, after an open(2) has been accomplished with the O_NONBLOCK flag, other I/O operations are also subject to the non-blocking rule. This is explained further in upcoming sections.

The following shows how a process can open its terminal line in non-blocking I/O mode:

int fd;                  // Terminal file descriptor

fd = open("/dev/tty",O_RDWR|O_NONBLOCK);
if ( fd == -1 ) {
    perror("open(2)");   // Report error
    abort();             // Abort run.
}
// fd is open in non-blocking I/O mode

Once the file descriptor is open in this manner, a call to read(2) will no longer suspend the program's execution while waiting for input.

Setting Non-Blocking Mode

Another method of choosing non-blocking I/O mode is to call upon the services of fcntl(2) after the file or device is already open:

#include <fcntl.h>
int fcntl(int fd, int cmd, ...);

where cmd is one of the following:

The command F_SETFL allows you to enable the flag O_NONBLOCK after the file has been opened. However, to do this, you will usually want to use the command F_GETFL to obtain the current flags in effect.

The following example shows how to enable O_NONBLOCK on an open file descriptor fd:

int fd;     /* Open file descriptor */
int fl;     /* Flags for fd */

fl = fcntl(fd,F_GETFL,0);
if ( fl == -1 ) {
    perror("fcntl(F_GETFL)");    /* Report failure */
    exit(13);
}

if ( fcntl(fd,F_SETFL,fl|O_NONBLOCK) == -1 ) {
    perror("fcntl(F_SETFL)");    /* Report failure */
    exit(13);
}

Notice how the flag O_NONBLOCK was ORed with the flags received in variable fl in the call to fcntl(2) using the F_SETFL command.

Performing Non-Blocking I/O

Once the file descriptor is in non-blocking I/O mode, you can use it with regular calls to read(2) and write(2). When no input is ready to be returned by read(2) or no output can be written by write(2), the returned error code in errno will be EAGAIN.

Listing 16.1 presents a program that uses non-blocking I/O on a FIFO.

Code Listing 16.1. nblockio.c—A Program That Reads a FIFO in Non-Blocking I/O Mode

1:   /* nblockio.c */
2:  
3:   #include <stdio.h>
4:   #include <unistd.h>
5:   #include <fcntl.h>
6:   #include <errno.h>
7:  
8:   int
9:   main(int argc,char **argv) {
10:      int z;          /* # of bytes returned */
11:      int fd;         /* File descriptor */
12:      char buf[256];  /* I/O buffer */
13: 
14:      fd = open("./fifo",O_RDWR|O_NONBLOCK);
15:      if ( fd == -1 ) {
16:          perror("open(2)");
17:          exit(13);
18:      }
19: 
20:      while ( (z = read(fd,buf,sizeof buf)) == -1 && errno == EAGAIN )
21:          ;
22: 
23:      if ( z >= 0 ) {
24:          buf[z] = 0;
25: 
26:          printf("GOT INPUT! '%s'
",buf);
27:      }  else
28:          perror("read(2)");
29: 
30:      return 0;
31:  }

Compiling the program with the make(1) file provided also creates this FIFO:

$ make nblockio
mkfifo ./fifo
cc -c  -Wall nblockio.c
cc -o nblockio nblockio.o
$

The program in Listing 16.1 opens the FIFO in line 14 in non-blocking mode (note the flag O_NONBLOCK). Once the FIFO is open, the program loops in line 20 as long as the error EAGAIN is returned from the read(2) call. The error EAGAIN tells the caller that no input is available for reading.

Once input is returned, the loop is exited, and the error or the data is reported in lines 23–28. The loop in lines 20–21 is very unfriendly to the system, and it will consume all available CPU trying to obtain input. However, in a real product, there would be other program events being performed in this loop instead.

Run the program in the background, so that you can use another command to put input into the FIFO. The following shows a sample session:

$ ./nblockio &
$ echo BOO >./fifo
$ GOT INPUT! 'BOO
'

[1] 19449 Exit 0              ./nblockio
$

The first command starts the program nblockio and places it in the background. At this point, it is chewing up CPU because of its non-blocking I/O loop.

The echo command is entered to feed the letters BOO and a linefeed character into the FIFO ./fifo, which the program is trying to read. Once that is done, the nblockio program reports that it got input, and it exits. You will need to press Return again to cause the job termination status to appear. The session output demonstrates that the nblockio program did read the input that was written to the FIFO.

The Problem with Non-Blocking I/O

The preceding demonstration shows how non-blocking I/O could be applied. However, if you were to run the program again and watch the system CPU usage with a resource-monitoring tool such as top(1), you would immediately recognize that the nblockio program was not a good UNIX citizen. It was using as much CPU as it could obtain from the kernel (this may not be as extreme, if you have other program functions to perform within the loop).

You would be forced to avoid using CPU time by calling a function such as sleep(3). Even if you use a more fine-grained timer such as nanosleep(2), you as the server designer will always be forced to compromise between latency and CPU overhead. As the sleep time is increased, the latency increases. As the sleep time is reduced, the CPU overhead increases.

An ideal solution for both your server and the rest of the host is to have your process awakened at the right time by the UNIX kernel. The kernel knows when it has data for your process to read on one of its open file descriptors. The kernel also knows when it can accommodate a write to one of the file descriptors belonging to your process.

In this fashion, the kernel suspends your server process from executing until there is something for it to perform. This allows precious CPU time to be used by other processes while your server process waits for something to happen. The kernel will awaken your process the moment it has pending I/O to perform. This is how efficiency is maintained within the host system while keeping server latency to a minimum.