This recipe is similar to the first, except this time you will create a data model for application logs. You will use Splunk's Data Model Editor to do this and will define a number of object types, and add constraints and attributes. In order to save pages, this recipe will be lighter on screenshots than the first recipe. The first recipe should therefore be used as a reference where needed.
To step through this recipe, you will need a running Splunk Enterprise server, with the sample data loaded from Chapter 1, Play Time – Getting Data In. You should have also completed the recipes from the earlier chapters. You should have also completed the first recipe in this chapter and be familiar with navigating the Splunk user interface.
Follow the steps in this recipe to create the Application data model:
- Log in to your Splunk server.
- Select the Operational Intelligence application.
- Select the Settings menu item at the top-right corner of the screen, and then select Data Models.
- The Web Access data model we created in the earlier recipe will be listed. Click on the New Data Model button located at the top-right corner of the screen.
- A pop-up box will be displayed. Enter
Applicationin the Title field and as you type, the ID will automatically populate. Ensure the selected app is Operational Intelligence, and then click on the Create button. - This will create an empty data model and the Data Model Editor will be displayed. Select the Add Object drop-down and select Root Event.
- The Add Event Object page will be displayed. Enter
All Applicationin the Object Name field and in the Constraints box, enter the search syntaxindex=main sourcetype=log4j. Once entered, click on the Preview button and some application log events will be displayed in the preview area. Following this, click on Save to save the event object type.
- After saving the object type, the Data Model Editor will be displayed with the newly created All Application object. Some inherited attributes will be seen on the right-hand side. You will now add a few more. Click on the Add Attribute drop-down and select Auto-Extracted.
- A pop-up box will be displayed with all the fields that Splunk has already extracted for the application logs. Check the checkbox next to the Field column heading at the top of the field list. Note that all the fields will then become checked. Once complete, click on Save, and all these fields will now become attributes for the data model object.
- You will now add a regular expression (regex) attribute to extract a new field called Service from the dataset. This regex matches a pattern in the event that relates to the different service events, either
perf,odbc, orshopapplication events. Click on the Add Attribute drop-down again, but this time select Regular Expression.
- The Add Attributes with a Regular Expression screen will be displayed. In the Regular Expression box, enter the following search syntax:
(?<Service>w+)(?=])
- Enter
Servicein the Field Name field if not already automatically populated and leave all the other fields with the defaults. Then click on the Preview button. You should see the new Service field populated in the preview results. Click on Save to save this new regular expression attribute.
- You will now create a number of nested child object types to help filter the different types of events within the application dataset. The following table lists all the child objects that you should create under the All Application Data root event object:
Child object
Secondary child object
Tertiary child object
Constraints
PerformanceService="perf"MemoryperfType="MEMORY"DBperfType="DB"DatabaseService="odbc"ShopService="shop"RequestrequestType=*ResponseNOT requestType=*Successresult="success"ErrorNOT result="success"Remember, that in order to add a child object, you need to select the Add Object drop-down and then select Child. Additionally, ensure that you preview the data as you go to confirm you have typed the attribute names correctly and that you have the child objects under the correct parent object!
- Once completed, your object hierarchy should resemble the following screenshot:
- Finally, we will add a transaction object type to group the event requests with respective event responses. Select the Add Object drop-down and select Root Transaction.
- The Add Transaction Object screen will be displayed. Enter
Transactionsin the Object Name field. Under Group Objects, select to group the Request and Response child objects that we just created. Select the threadid object attribute in the Group by field and enter a maximum span of1hour. - Click on the Preview button and you should see the grouped transactions populated in the event results box. Click on Save to save this new object type.
- Our Application data model is now complete—congratulations!
In this recipe, you started off replicating a similar path to the first recipe by creating a new data model for our application dataset. After the data model was created, you added a root-level event object type, named All Application, to sit at the top of the object hierarchy. This event object allows simple constraints and you created an object constraint that constrained the object to application logs. Following this, you added object attributes to the object, consisting of all the autoextracted fields that Splunk already knew about, in addition to a regular expression attribute to categorize the various services within the event data. You then used this newly created Service regular expression attribute plus the other Auto-Extracted attributes to create several nested child objects in order to build an object hierarchy for the Application data model. We also added a root level transaction object type, which grouped related application events into individual transactions based on a common threadid.
Behind the scenes, Splunk is essentially creating a Splunk search to report on the dataset that is being modeled. The constraints provided actually tell Splunk what data to look at and the attributes are basically the fields within the data that Splunk will search. The child object types inherit all the attributes and constraints from their parents and act as further filters for the backend search that Splunk creates.