It is a good idea to understand the underlying performance of the hard disks and SSDs in your Ceph cluster, as this will enable you to predict the overall performance of your Ceph cluster. To benchmark the disks in your cluster, the fio tool will be used.

Fio is a complex tool with many configuration options. For the purpose of this chapter, we will concentrate on using it to perform basic read and write benchmarks:

1. Install the fio tool on a Ceph OSD node:

       apt-get install fio

The preceding command gives the following output:

2. Now, create a new file and place the following fio configuration into it:

       [global]
       ioengine=libaio
       randrepeat=0
       invalidate=0
       rw=randwrite
       bs=4k
       direct=1
       time_based=1
       runtime=30
       numjobs=1
       iodepth=1
       filename=/test.fio
       size=1G

The previous fio config will run a single threaded 4 KB random write test for 30 seconds. It will create a 1G test.fio file in the root of the filesystem. If you wish to target a block device directly, simply set the filename to the block device; but note that with the preceding warning, fio will overwrite any data on that block device.

Notice that the job is set to use direction, so the page cache will not accelerate any I/O operations.

To run the fio job, simply call fio with the name of the file to which you saved the previous config:

    fio <filename>

The preceding command gives the following output:

Once the job is done, fio will produce an output similar to the previous screenshot. You can see that the fio job runs 39 IOPS and 162 MBps on an average, and the average latency was 25 milliseconds.

There is also a breakdown of latency percentiles, which can be useful for understanding the spread of the request latency.