When a file is transferred by an application, the kernel first buffers the data and then sends the data to the application buffers. The application, in turn, sends the data to the destination. The Sendfile method is an improved method of data transfer, in which data is copied between file descriptors within the OS kernel space, without transferring data to the application buffers. This results in the improved utilization of the operating system's resources.

The method can be enabled using the sendfile directive. The directive is available for the http, server, and location sections:

http{
  sendfile on;
}

The flag is set to off by default.

The OS kernel usually tries to optimize and cache any read/write requests. Since the data is cached within the kernel, any subsequent read request to the same place will be much faster because there's no need to read the information from slow disks.

Direct I/O is a feature of the filesystem where reads and writes go directly from the applications to the disk, thus bypassing all OS caches. This results in better utilization of CPU cycles and improved cache effectiveness.

The method is used in places where the data has a poor hit ratio. Such data does not need to be in any cache and can be loaded when required. It can be used to serve large files. The directio directive enables the feature. The directive is available for the http, server, and location sections:

location /video/ {
  directio 4m;
}

Any file that's larger than that specified in the directive will be loaded by direct I/O. The parameter is disabled by default.

Direct I/O depends on the block size while doing a data transfer. NGINX has the directio_alignment directive to set the block size. The directive is present under the http, server, and location sections:

location /video/ {
  directio 4m;
  directio_alignment 512;
}

The default value of 512 bytes works well for all boxes unless it is running a Linux implementation of XFS. In such an instance, the size should be increased to 4 KB.

Asynchronous I/O allows a process to initiate I/O operations without having to block or wait for it to complete.

The aio directive is available under the http, server, and location sections of an NGINX configuration. Depending on the section, the parameter will perform asynchronous I/O for the matching requests. The parameter works on Linux kernel 2.6.22+ and FreeBSD 4.3. The following code shows this:

location /data {
  aio on;
}

By default, the parameter is set to off. On Linux, aio needs to be enabled with directio, while on FreeBSD, sendfile needs to be disabled for aio to take effect.

The directive has a special value of threads, which enables multithreading for send and read operations. The multithreading support is only available on the Linux platform and can only be used with the epoll, kqueue, or eventport methods of processing requests.

In order to use the threads value, configure multithreading in the NGINX binary using the --with-threads option. Post this, add a thread pool in the NGINX global context using the thread_pool directive. Use the same pool in the aio configuration:

thread_pool io_pool threads=16;
http{
….....
   location /data{
     sendfile    on;
     aio        threads=io_pool;
   }
}

The three directives can be mixed together to achieve different objectives on different platforms. The following configuration will use sendfile for files that are smaller than what is specified in directio. Files served by directio will be read using asynchronous I/O:

location /archived-data/{
  sendfile on;
  aio on;
  directio 4m;
}

The aio directive has a sendfile value, which is available only on the FreeBSD platform. The value can be used to perform Sendfile in an asynchronous manner:

location /archived-data/{
  sendfile on;
  aio sendfile;
}

NGINX invokes the sendfile() system call, which returns with no data in the memory. Post this, NGINX initiates data transfer in an asynchronous manner.