In Splunk, for any kind of analytics and visualizations, fields play a very important role. Splunk automatically tries to extract and make them available for use for known and properly configured data sources. Since there are a wide variety of sources for data, there could be many fields which do not get automatically extracted. Splunk also provides the Splunk command rex, which can be used to extract the fields, but this command requires a good understanding of regular expressions to efficiently extract fields from the data. So Splunk provides a very easy to use field extractor to extract fields using an interactive field extractor tool via the Splunk Web interface.
Let us learn to access the field extractor to extract fields from the data, which in turn can be used to create analytics and visualizations in Splunk.
The field extractor can be accessed via the following options:
- Splunk Web Console | Settings | Fields | Field Extractions | Open Field Extractor.
- Using direct URL, by navigating to
http://localhost:8000/en-US/app/launcher/field_extractorWhere
localhostand8000are to be replaced with, respectively, the IP address and web port of the user's Splunk instance. - This option to access the field extractor is one the most useful and recommended ways of accessing the field extractor. It can be accessed from the result window of events when a search query is run on the web console.
The preceding image shows an example to extract the field extractor tool, which can be accessed by taking the following steps:
- Open a Splunk search page via the web console.
- Query the data which has the value to be extracted as a field.
- Click on the downward pointing arrow option of the respective event to expand the event options.
- Click on the Event Actions button on the expanded window.
- Then click on the Extract Fields option to extract the field from the selected data.
Now, since we are aware of accessing the field extractor tool, let us see how to extract the fields using this tool. We will learn field extraction using the easiest and most recommended option, which is the third option explained in the Accessing field extractor section.
On clicking Extract fields via the third option, the following interactive screen appears. Even if the first and second options are chosen to access the field extractor, after selecting sourcetype, a screen similar to the following is shown:
The preceding image has the following components:
- Navigation panel: The panel to proceed further and backtrack if required during the whole process of field extraction.
- Data: A subset of data of the selected sourcetype is shown, giving a choice of which option to choose (Regular Expression or Delimiters) for further field extraction.
- Regular Expression: If the field which is to be extracted from the data is based on regular expression, then this option is selected, that is, field extraction using regular expression.
- Delimiters: If the field which is to be extracted is from a structured file like CSV or TSV, then this option is selected.
Let us have a look at field extraction using an example via the Splunk Web console. First we will learn field extraction using the first option as in the preceding image, that is, field extraction using regular expression, and then using the second option, or using delimiters.
Let us have a look at field extraction using regular expression. When regular expression is selected, then a screen similar to the following appears where we can extract the required fields:
- In the Select Fields where the data is shown, a set or subset of data can be selected to extract as a field. Let us say if the user wants to extract the IP address, then select the IP address and specify a name as required to get extracted. The following image describes the same:
- Click on Add Extraction and it will create a field as per the specified name and preview the value from the selected sourcetype, and verify and correct it if there is any discrepancy in extracting the value.
- The following image shows the preview of the extracted field (Recievers_IPAddress):
The Preview window has the option to select and deselect events to review any event which is wrongly extracted or any event which has a similar regular expression but is not extracted. Various options exist, like previewing matches and non-matches of fields from the data. The event selection sample can also be selected from the Preview window.
- Once the required events are properly extracted, then click on Next to validate and save the newly extracted field.
- The preceding procedure to extract the fields can be done for any number of datasets, and the required information can be extracted in the form of fields. Now let's suppose the user is interested in the IP address of only mails which were received in the inbox. Since the sourcetype will include all the data of the mail box, it will contain the IP address of various sent and received e-mails. So if any other conditions are to be specified while extracting the field, then it can be done here.
The following image shows how we can extract only the IP address of e-mails which are received in the mailbox:
On selecting the Received option as shown in the preceding image, there are two options available. One is to extract similar data patterns in a field and another one is the Require option. It is used to instruct the field extractor tool to extract only those IP addresses (regular expression) where a received value is available. Thus, now the Recievers_IPAddress field will contain only those IP addresses where Received is available in the data.
Similarly, this tool can be used to extract fields based on requirements and can be used to build analytics and visualizations in Splunk.
When the delimiter is chosen while extracting the fields then a screen similar to the following appears:
From the preceding screen, the respective delimiter option, when selected, automatically extracts all the data into fields; as per requirements, the field can be renamed and used. In the preceding example image, it is clearly seen that the data is comma-separated, and hence we will use a comma delimiter to properly extract the fields.
The following image shows the field extraction when the comma delimiter is selected for the preceding example:
Thus, fields can be extracted on the basis of regular expression as well as on the basis of delimiter, as illustrated with the preceding example. Once fields are extracted, click on Next and then Save to make the extracted fields available for analytics and visualization. Permission to save extracted fields for any single app or for all the apps can be configured before saving right from the field extractor tool itself.