Creating a custom search command to format product names

Sometimes, you just need that extra bit of logic or custom processing of data that might be very unique to your line of business. You might also simply be in a position where you have picky executives who like to see their data formatted in a very specific manner.

In this recipe, you will learn how to use Splunk's Python SDK to create a custom search command that you can use to apply consistent formatting to product names or any other string field, by capitalizing the first letter of each word in a string.

Getting ready

To step through this recipe, you will need a running Splunk Enterprise server, with the sample data loaded from Chapter 1, Play Time – Getting Data In. You should be familiar with navigating the Splunk user interface and using the Splunk search language. Some basic knowledge of Python is recommended. The Splunk Python SDK should also be downloaded and available on your Splunk Enterprise server.

How to do it...

Perform the steps in this recipe to create a custom search command to format product names:

  1. Open a console terminal on your Splunk server.
  2. Change to the directory where you downloaded the Splunk Python SDK.
  3. Expand the ZIP file using an appropriate tool located on your Splunk server.
  4. Create a splunklib directory inside the $SPLUNK_HOME/etc/apps/ operational_intelligence/bin directory.
  5. Copy the splunk-sdk-python/splunklib directory to $SPLUNK_HOME/etc/apps/operational_intelligence/bin/splunklib.
  6. Create and add the following code to commands.conf, located in the $SPLUNK_HOME/etc/apps/operational_intelligence/local directory:
     [fixname]
filename = fixname.py
supports_getinfo = true
supports_rawargs = true
outputheader = true
requires_srinfo = true
  7. In $SPLUNK_HOME/etc/apps/operational_intelligence/bin, create fixname.py and add the following code:
    #!/usr/bin/env python

import sys

from splunklib.searchcommands import 
    dispatch, StreamingCommand, Configuration, Option, validators

@Configuration()
class FixNameCommand(StreamingCommand):
    """ Takes the first letter of each word in the field and capitalizes it
    
    ##Syntax

    .. code-block::
        fixname fieldname=<field>

    ##Description

    Takes the first letter of each word in the field and capitalizes it

    ##Example

    Uppercase the first letter of each word in the message field in the _internal
    index

    .. code-block::
        index=_internal | head 20 | fixname fieldname=message

    """
    fieldname = Option(
        doc='''
        **Syntax:** **fieldname=***<fieldname>*
        **Description:** Name of the field that will be capitalized''',
        require=True, validate=validators.Fieldname())

    def stream(self, records):
        self.logger.debug('FixNameCommand: %s' % self)  # logs command line
        for record in records:
            record[self.fieldname] = record[self.fieldname].title()
            yield record

dispatch(FixNameCommand, sys.argv, sys.stdin, sys.stdout, __name__)
  8. Ensure that the fixname.py script is marked as an executable by executing the following command:
    chmod a+x fixname.py
  9. Restart Splunk.
  10. Log in to Splunk.
  11. Select the Operational Intelligence application.
  12. In the search bar, enter the following search over Last 24 hours:
    index=main sourcetype=log4j ProductName=* | eval ProductName=lower(ProductName) | fixname fieldname=ProductName

    You should see that despite forcing the ProductName field values to be all lowercase, the fixname command has now capitalized each value.

How it works...

The Splunk Python SDK can allow us to not only get information out of Splunk in an easy, programmatic way, but also manipulate the processing of events as they move through our search.

Originally, custom search commands could be created using Python and added to Splunk, but they were difficult to debug and had no logging mechanism. With the Python SDK, you can now create your own custom search commands in a quicker and easier way with better tools for troubleshooting.

Custom search commands come in three different flavors:

Command

Description

Generating commands

This type of command generates new events that are inserted into the results. Examples include commands that read from lookup files, such as inputcsv.

Reporting commands

This type of command takes incoming events and generates a new set of outgoing events, usually based on some sort of processing or analysis. Examples include commands that do statistics, such as stats and top.

Streaming commands

This type of command takes incoming events and modifies or filters outgoing events. Examples include commands that add or replace fields or eliminate events based on some calculations, such as eval, rename, and where.

Let's explain how the fixname.py script works:

Script fragment

Description

#!/usr/bin/env python

import sys

from splunklib.searchcommands import 
    dispatch, StreamingCommand, Configuration, Option, validators

Import the necessary modules and libraries. This includes the Splunk library that has to be copied to the bin directory of the Splunk app.

@Configuration()

Here, we apply any configuration options that need to be specified to Splunk when the command is executed.

class FixNameCommand(StreamingCommand):

This line defines the class name of the command as well as any inheritance that might be required. In this command, the StreamingCommand class is to be inherited from.

""" Takes the first letter of each word in the field and capitalizes it
    
    ##Syntax

    .. code-block::
        fixname fieldname=<field>

    ##Description

    Takes the first letter of each word in the field and capitalizes it

    ##Example

    Uppercase the first letter of each word in the message field in the _internal
    index

    .. code-block::
        index=_internal | head 20 | fixname fieldname=message

    """

Here, we outline all of the information that Splunk will present through the Splunk web interface in the search bar help.

    fieldname = Option(
        doc='''
        **Syntax:** **fieldname=***<fieldname>*
        **Description:** Name of the field that will be capitalized''',
        require=True, validate=validators.Fieldname())

This section defines the various options that the custom command will accept or is required to accept. The format as well as any validation that is required is also specified here.

def stream(self, records):
        self.logger.debug('FixNameCommand: %s' % self)  # logs command line
        for record in records:
            record[self.fieldname] = record[self.fieldname].title()
            yield record

This section implements the stream function. The stream function is called when records are to be processed. In this example, we iterate through each of the records, and depending on the field that was defined in the options, we execute the title method on that value.

dispatch(FixNameCommand, sys.argv, sys.stdin, sys.stdout, __name__)

Finally, we dispatch the command, passing in the required arguments.

The fixname command is a straightforward command that leverages the title operation, available within the String object in Python, to uppercase the string that is in the requested field. It is a streaming command, because it manipulates a field within an event as it moves through the command.

By leveraging the SDK, any number of commands can be developed that integrate with third-party systems or apply proprietary algorithms or logic to implement business rules that give organizations better visibility of their operations.

Note

For more information on how to create custom search commands, check out the documentation at http://dev.splunk.com.

See also

  • The Remotely querying Splunk's REST API for unique page views recipe
  • The Creating a Python application to return unique IP addresses recipe
..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset
3.138.105.31