This section is all about configuring outputs where the events need to be shipped. Events can be sent to single or multiple outputs simultaneously. The allowed outputs are Elasticsearch, Logstash, Kafka, Redis, file, and console. Some outputs that can be configured are as follows:

• elasticsearch: This is used to send events directly to Elasticsearch. A sample Elasticsearch output configuration is shown in the following code snippet:

output.elasticsearch:
  enabled: true  
  hosts: ["localhost:9200"]
  

Using the enabled setting, you can enable or disable the output. hosts accepts one or more Elasticsearch node/servers. Multiple hosts can be defined for failover purposes. When multiple hosts are configured, the events are distributed to these nodes in a round-robin order. If Elasticsearch is secure, then credentials can be passed using the username and password settings, as follows:

output.elasticsearch:
  enabled: true  
  hosts: ["localhost:9200"]
  username: "elasticuser"
  password: "password"

To ship events to the Elasticsearch ingest node pipeline so that they can be preprocessed before being stored in Elasticsearch, pipeline information can be provided using the pipleline setting, as follows:

output.elasticsearch:
  enabled: true  
  hosts: ["localhost:9200"]
  pipeline: "ngnix_log_pipeline"

The default index the data gets written to is in the metricbeat-%{[beat.version]}-%{+yyyy.MM.dd} format. This will create a new index every day. For example, if today is April 02, 2019, then all the events are placed in the metricbeat-7.0.0-2019-04-02 index. You can override the index name or the pattern using the index setting. In the following configuration snippet, a new index is created for every month, as follows:

output.elasticsearch:
  hosts: ["http://localhost:9200"]
  index: "metricbeat-%{[beat.version]}-%{+yyyy.MM}"

Using the indices setting, you can conditionally place the events in the appropriate index that matches the specified condition. In the following code snippet, if the message contains the DEBUG string, it will be placed in the debug-%{+yyyy.MM.dd} index. If the message contains the ERR string, it will be placed in the error-%{+yyyy.MM.dd} index. If the message contains neither of these strings, then those events will be pushed to the logs-%{+yyyy.MM.dd} index, as specified in the index parameter, as follows:

output.elasticsearch:
  hosts: ["http://localhost:9200"]
  index: "logs-%{+yyyy.MM.dd}"
  indices:
    - index: "debug-%{+yyyy.MM.dd}"
      when.contains:
        message: "DEBUG"
    - index: "error-%{+yyyy.MM.dd}"
      when.contains:
        message: "ERR"

• logstash: This is used to send events to Logstash.

A sample Logstash output configuration is as follows:

output.logstash:
  enabled: true  
  hosts: ["localhost:5044"]

Using the enabled setting, you can enable or disable the output. hosts accepts one or more Logstash servers. Multiple hosts can be defined for failover purposes. If the configured host is unresponsive, then the event will be sent to one of the other configured hosts. When multiple hosts are configured, events are distributed in a random order. To enable load-balancing events across the Logstash hosts, use the loadbalance flag, set to true, as follows:

output.logstash:
  hosts: ["localhost:5045", "localhost:5046"]
  loadbalance: true

• console: This is used to send events to stdout. These events are written in JSON format. This is useful during debugging or testing.

A sample console configuration is as follows:

output.console:
  enabled: true
  pretty: true