BlueStore can journal writes in the RocksDB WAL and flush them at a later date, allowing for write coalescing and ordering. This can bring large performance improvements for clusters that use spinning disks with flash-based devices for RocksDB.

By default, if the OSD is identified as a spinning HDD, writes less or equal to 32 KB are written into the WAL of the OSD and are then acknowledged and sent back to the client. This is controlled by the bluestore_prefer_deferred_size_hdd variable; this value can be adjusted if it is determined that your workload would benefit from also deferring larger writes via the WAL to achieve lower latency and higher IOPS. Thought should also be given to the write load of the flash device holding the WAL, both for bandwidth and endurance reasons.

The BlueStore configuration also limits how many writes can be queued up before the OSD is forced to flush them down to the disk; this can be controlled via the bluestore_deferred_batch_ops variable and is set by default to 64. Increasing this value may increase total throughput, but also runs the risk of the HDD spending large amounts of time being saturated and raising the average latency.