The Splunk Enterprise stores its index's data into buckets organized by age. Basically, it is a directory containing events of a specific period. There can be several buckets at the same time in the various stages of the bucket life cycle.
A bucket moves from one stage to another depending upon its age, size, and so on, as per the defined conditions. The Splunk bucket stages are Hot, Warm, Cold, Frozen, and Thawed. Splunk buckets play a very important role in the performance of search results and hence they should be properly configured as per the requirements.
The following image shows the life cycle of Splunk buckets:
Let us understand the Splunk bucket life cycle, taking the above image as a reference. The Indexes.conf file can be modified to configure the aging and the conditions to move from one stage to another:
- Hot bucket: Whenever any new data gets indexed on Splunk Enterprise, it is stored in a hot bucket. There can be more than one hot bucket for each index. The data in the hot bucket supports both read and write. This is the only stage of the bucket life cycle where it supports
writeoperations as well. Until and unless some specific conditions are configured, the data in the hot bucket cannot be backed up. - Warm bucket: Whenever the hot bucket is full, it gets converted into warm bucket and a new hot bucket gets created. Unlike hot bucket, the data in the warm bucket only supports read and can be backed up. In terms of search performance, hot and warm Buckets are the same, with no effect on search performance. Hot and warm buckets are stored at
$SPLUNK_HOME/var/lib/splunk/defaultdb/db/* - Cold bucket: Once the warm bucket is full or the count of the warm bucket exceeds the configured number, the warm bucket is moved to the cold bucket. The storage type used for the cold bucket can be relatively cheaper as compared to that of the hot/warm bucket. The hot/warm bucket requires very high IOPS as compared to the data in the cold bucket and hence, relatively cheaper storage can be used for the cold bucket. Similar to the warm bucket, it supports both read and backup capability. The cold bucket is stored at
$SPLUNK_HOME/var/lib/splunk/defaultdb/colddb/* - Frozen bucket: On reaching the age limit or crossing the storage limit of the cold bucket, the cold bucket is converted into the frozen bucket. Frozen data does not support
readoperations and cannot be searched on either. Splunk, by default, deletes the frozen bucket but it can be configured to move to an archive as well. Archived data can later move to the thawed state.