Configuring Filebeat

Configurations related to Filebeat are stored in a configuration file named filebeat.yml. They use the YAML syntax. 

The filebeat.yml file contains the following important sections:

  • Filebeat inputs
  • Filebeat modules
  • Elasticsearch template settings
  • Filebeat general/global options
  • Kibana dashboard settings
  • Output configuration 
  • Processors configuration 
  • Logging configuration
The filebeat.yml file will be present in the installation directory if .zip or .tar files are used. If dep or rpm is used for installation, then it will be present in the /etc/filebeat location.

Some of these sections are common for all type of Beats. Before we look into some of these, let's see what a simple configuration would look like. As we can see in the following configuration, when Filebeat is started, it looks for files ending with the .log extension in the E:packtlogs path. It ships the log entries of each file to Elasticsearch, which is configured as the output, and is hosted at localhost:9200:

#filebeat.yml
#=========================== Filebeat inputs =============================

filebeat.inputs:

- type: log

# Change to true to enable this input configuration.
enabled: true

# Paths that should be crawled and fetched. Glob based paths.
paths:
- E:packtlogs*.log

#================================ Outputs =====================================

#-------------------------- Elasticsearch output ------------------------------
output.elasticsearch:
# Array of hosts to connect to.
hosts: ["localhost:9200"]
Any changes made to filebeat.yml require restarting Filebeat to pick up the changes.

Place some log files in E:packtlogs. To get Filebeat to ship the logs, execute the following command:

Windows:
E:>filebeat-7.0.0-windows-x86_64>filebeat.exe

Linux:
[locationOfFilebeat]$ ./filebeat


To run the preceding example, please replace the content of the default filebeat.yml file with the configuration provided in the preceding snippet.

To validate whether the logs were shipped to Elasticsearch, execute the following command:

E:>curl -X GET http://localhost:9200/filebeat*/_search?pretty

Sample Response:
{
"took" : 2,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 3,
"relation" : "eq"
},
"max_score" : 1.0,
"hits" : [
{
"_index" : "filebeat-7.0.0-2019.04.22",
"_type" : "_doc",
"_id" : "bPnZQ2oB_501XGfHmzJg",
"_score" : 1.0,
"_source" : {
"@timestamp" : "2019-04-22T07:01:30.820Z",
"ecs" : {
"version" : "1.0.0"
},
"host" : {
"id" : "254667db-4667-46f9-8cf5-0d52ccf2beb9",
"name" : "madsh01-I21350",
"hostname" : "madsh01-I21350",
"architecture" : "x86_64",
"os" : {
"platform" : "windows",
"version" : "6.1",
"family" : "windows",
"name" : "Windows 7 Enterprise",
"kernel" : "6.1.7601.24408 (win7sp1_ldr_escrow.190320-1700)",
"build" : "7601.24411"
}
},
"agent" : {
"type" : "filebeat",
"ephemeral_id" : "d2ef4b77-3c46-4af4-85b4-e9f690ce00f1",
"hostname" : "madsh01-I21350",
"id" : "29600459-f3ca-4516-8dc4-8a0fd1bd6b0f",
"version" : "7.0.0"
},
"log" : {
"offset" : 0,
"file" : {
"path" : "E:\packt\logs\one.log"
}
},
"message" : "exception at line1",
"input" : {
"type" : "log"
}
}
},
...
...
...

 

Filebeat places the shipped logs under an filebeat index, which is a time-based index based on the filebeat-YYYY.MM.DD pattern. The log data would be placed in the message field.

To start Filebeat on deb or rpm installations, execute the 
sudo service filebeat start command. If installed as a service on Windows, then use Powershell to execute the following command:

PS C:> Start-Service filebeat
..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset
18.117.217.186