In this chapter, we will introduce the Splunk data model and pivoting functionality. We will cover the following recipes:
- Creating a data model for web access logs
- Creating a data model for application logs
- Accelerating data models
- Pivoting total sales transactions
- Pivoting purchases by geographical location
- Pivoting slowest responding web pages
- Pivot charting top error codes
In many of the previous chapters, we leveraged Splunk's SPL quite a bit in order to build searches, reports, and dashboards. In this chapter, we will learn how to leverage Splunk's data model and Pivot functionality, and demonstrate how these can be leveraged by less technical users to easily build reports, charts, and dashboards.
The first set of recipes in this chapter involves building Splunk data models. Data models allow Splunk datasets to be mapped, together with associated knowledge, into a hierarchical structure that encapsulates a number of Splunk searches behind the scenes. These models power Splunk's Pivot tool and allow the users to create dynamic reports and dashboards, without the need to write any searches. Data models are somewhat analogous to relational database schemas; in that, they present data to Pivot as rows and columns.
Data models are typically built by individuals who are familiar with Splunk's SPL using the Data Model Editor. Data models have a hierarchical structure made up of objects, object types, object constraints, and object attributes. A data model consists of one or more objects and each object is a certain object type. Each object has one or more object constraints and contains one or more object attributes.
There are four different object types and these are outlined in the following table:
All data model objects are defined by one or more object constraints which help filter out irrelevant events, and these are outlined in the following table:
All data model objects have one or more object attributes. These are basically fields within the dataset being modeled, and the available attribute types are outlined in the following table:
Tip
The Common Information Model add-on app contains a number of predefined data models that comply with Splunk's Common Information Model (CIM). The CIM add-on can be used when modeling data to ensure compatibility or to take advantage of the prebuilt data models to Pivot and report. Also, many of Splunk's premium apps, such as Enterprise Security, leverage the CIM. The add-on is available for download at http://splunkbase.splunk.com/app/1621/.
Once we have built our data models in this chapter, you will learn how to accelerate them. An accelerated data model leverages Splunk's underlying High Performance Analytics Store (HPAS), building summaries alongside the buckets of data in the associated indexes, and allows for significant performance increases in Pivot-based reporting across extremely large datasets.
Note
For more information on data models, review the Knowledge Manager documentation at http://docs.splunk.com/Documentation/Splunk/latest/Knowledge/Aboutdatamodels.
The second half of this chapter is dedicated to using Splunk's Pivot tool in order to search and report on the data we have modeled. Pivot enables the users to report on data within Splunk, without having to use the SPL. The Pivot interface provides drag-and-drop functionality, allowing for easy reporting and visualization of Splunk datasets. As Pivot leverages data models and their associated objects for reporting, a data model must be created before Pivot can be used.
The fundamentals of data models and Pivot can be quite challenging to get to grips with initially and there is no better way to learn than to get our hands dirty and start modeling and pivoting in Splunk; so, let's do that!