Mutate filter

You can perform general mutations on fields using this filter. The fields in the event can be renamed, converted, stripped, and modified.

Let's enhance the csv_file.conf file we created in the previous section with the mutate filter and understand its usage. The following code block shows the use of the mutate filter:

#csv_file_mutuate.conf
input {
file{
path => "D:eslogsusers.csv"
start_position => "beginning"
sincedb_path => "NULL"
}

}

filter {
csv{
autodetect_column_names => true
}

mutate {
convert => {
"Age" => "integer"
"Salary" => "float"
}
rename => { "FName" => "Firstname"
"LName" => "Lastname" }
gsub => [
"EmailId", ".", "_"
]
strip => ["Firstname", "Lastname"]
uppercase => [ "Gender" ]
}
}

output {
stdout {
codec => rubydebug
}
}

As we can see, the convert setting within the filter helps to change the datatype of a field. The valid conversion targets are integer, string, float, and boolean

If the conversion type is boolean, these are the possible values:
True: true, t, yes, y, and 1.
False: false, f, no, n, and 0.

The rename setting within the filter helps rename one or more fields. The preceding example renames the FName field to Firstname and LName to Lastname.

gsub matches a regular expression against a field value and replaces all matches with a replacement string. Since regular expressions work only on strings, this field can only take a field containing a string or an array of strings. It takes an array consisting of three elements per substitution (that is, it takes the field name, regex, and the replacement string). In the preceding example, . in the EmailId field is replaced with _.

Make sure to escape special characters such as , ., +, and ? when building regex.

strip is used to strip the leading and training white spaces. 

The order of the settings within the mutate filter matters. The fields are mutated in the order the settings are defined. For example, since the FName and LName fields in the incoming event were renamed to Firstname and Lastname using the rename setting, other settings can no longer refer to FName and LName. Instead, they have to use the newly renamed fields.

uppercase is used to convert the string into upper case. In the preceding example, the value in the Gender field is converted into upper case.

Similarly, by using various settings of the mutate filter, such as lowercase, update, replace, join, and merge, you can lower-case a string, update an exiting field, replace the value of a field, join an array of values, or merge fields. 

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset
18.219.102.189