Chapter 7. Enriching Data – Lookups and Workflows

In this chapter, we will learn how to augment and enrich the data within Splunk. You will learn about:

  • Looking up product code descriptions
  • Flagging suspect IP addresses
  • Creating a session state table
  • Adding hostnames to IP addresses
  • Searching ARIN for a given IP address
  • Triggering a Google search for a given error
  • Creating a ticket for application errors
  • Looking up inventory from an external database

Introduction

In the previous chapter, you continued to improve your Splunk search and analytical skills by creating highly advanced searches that leveraged more of the deep analytical commands to gain more operational intelligence from the data contained within the logs. In this chapter, you will leverage Splunk's lookup functionality to enrich these results with the data found outside of the logs. You will also use Splunk's workflow functionality to perform some simple actions on the data that you discovered.

Lookups

Lookups are used to enrich log data with additional data not found in the log events themselves. They allow you to key off one or more fields in the event data and add additional fields to this data. These additional fields are commonly added by looking up the specified fields in a static CSV-based lookup table and then bringing back additional fields associated with that specific entry in the table. However, lookups can also be a lot more dynamic, leveraging Python scripts, using Splunk's built-in Key-Value (KV) store, or directly looking up fields in an external database table. Lookup tables can also contain time fields to allow for time-based lookups in a given time period.

While external lookups using Python scripts are usually dynamic in nature, you might think that CSV-based lookups would only be useful for types of data such as HTTP status codes that never change over time (well, maybe every once in a while). However, Splunk can be used to populate CSV or KV store lookups using the output of a search and saving this as a lookup table. This technique can provide many different possible uses, from identifying trends to creating various aging reports. Splunk's KV store even allows for basic CRUD (Create, Read, Update, Delete) functionality on lookups. In this chapter, you will build and populate a number of lookups and leverage a dynamic scripted lookup.

Workflows

When working with events being returned by Splunk, there are often times when there is a need to perform a subsequent operation in order to get more details. Sometimes performing another search within Splunk is enough, but at other times, you might need to send this data to an external system for further processing.

Splunk provides a feature known as workflow action that can be configured to provide different options, depending on what fields are present in your search results. There are two types of workflow actions currently available: the ability to open a link to a web-based resource and the ability to execute an additional search within Splunk. The link action can be used to search for data in popular search engines or to link to other internal resources such as helpdesk tools or change management systems. The search action can be used to initiate more complex searches than you can get with a chart or table drilldown. In this chapter, you will build both link and web-based workflow actions.

DB Connect

Splunk DB Connect is a Splunk supported application for Splunk Enterprise that lets you enrich and combine your log data with external database data. Using DB Connect, you can query external databases using SQL from Splunk directly and return the results to Splunk. These results can be combined with other log data, converted to local lookups, or indexed into Splunk. DB Connect also allows for external database lookups to enrich your log data with additional information while it is being searched. At the end of this chapter, we will install DB Connect and pull in inventory data from an external database.

Ok, you are probably excited to begin, so let's get going!

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset
52.14.130.13