A Watch is made of the following components:

• schedule: This is used to specify the time interval for scheduling/triggering the watch.
• query: This is used to specify a query to retrieve data from Elasticsearch and run it as an input to the condition. Elasticsearch Query DSL/Lucene queries can be used to specify the queries.
• condition: This is used to specify conditions against the input data obtained from the query and check whether any action needs to be taken or not.
• action: This is used to specify actions such as sending an email, sending a slack notification, logging the event to a specific log, and much more on meeting the condition:

Let's look into a sample watch and understand the building blocks of a watch in detail. The following code snippet creates a watch:

curl -u elastic:elastic -X POST  http://localhost:9200/_xpack/watcher/watch/logstash_error_watch   -H 'content-type: application/json'   -d '{

  "trigger" : { "schedule" : { "interval" : "30s" }},
  "input" : {
    "search" : {
      "request" : {
        "indices" : [ "logstash*" ],
        "body" : {
          "query" : {
            "match" : { "message": "error" }
          }
        }
      }
    }
  },
  "condition" : {
    "compare" : { "ctx.payload.hits.total" : { "gt" : 0 }}
  },
  "actions" : {
    "log_error" : {
      "logging" : {
        "text" : "The number of errors in logs is  {{ctx.payload.hits.total}}"
      }
    }
  }
}'

• trigger: This section is used to provide a schedule to specify how often the watch needs to be executed. Once the watch is created, Watcher immediately registers its trigger with the scheduler trigger engine. The trigger engine evaluates the trigger and runs the watch accordingly.

Several types of schedule triggers can be defined to specify when watch execution should start. The different types of schedule triggers are interval, hourly, daily, weekly, monthly, yearly, and cron. 

In the preceding code snippet, a trigger was specified with a schedule of 30 seconds, which means that the watch is executed every 30 seconds.

Example to specify hourly trigger: The following snippet shows how to specify an hourly trigger that triggers the watch every 45th minute of an hour:

{
  "trigger" : {
    "schedule" : {
      "hourly" : { "minute" : 45 }
    }
  }
}

You can specify an array of minutes, too. The following snippet shows how to specify an hourly trigger that triggers the watch every 15th and 45th minute of an hour:

{
  "trigger" : {
    "schedule" : {
      "hourly" : { "minute" : [ 15, 45 ] }
    }
  }
}

The following is an example of specifying that the watch should trigger daily at 8 PM:

{
  "trigger" : {
    "schedule" : {
      "daily" : { "at" : "20:00" }
    }
  }
}

The following is an example of specifying a watch to trigger weekly on Mondays at 10 AM and on Fridays at 8 PM:

{
  "trigger" : {
    "schedule" : {
      "weekly" : [
        { "on" : "monday", "at" : "10:00" },
        { "on" : "friday", "at" : "20:00" }
      ]
    }
  }
}

The following is an example of specifying a schedule using cron syntax. The following snippet specifies a watch to be triggered hourly at the 45th minute:

{ 
  "trigger" : { 
    "schedule" : { 
      "cron" : "0 45 * * * ?" 
    } 
  } 
}

• input: This section is used to specify the input to load the data into the Watcher execution context. This data is referred to as Watcher Payload and will be available/accessible in subsequent phases of the watcher execution so that it can be used to create conditions on it or used when generating actions. The payload can be accessed using the ctx.payload.* variable:

"input" : {
    "search" : {
      "request" : {
        "indices" : [ "logstash*" ],
        "body" : {
          "query" : {
            "match" : { "message": "error" }
          }
        }
      }
    }
  }

As shown in the preceding code, an input of the search type is used to specify the query to be executed against Elasticsearch to load the data into Watcher Payload. The query fetches all the documents present in the indices of the logstash* pattern that contain error in the message field.

• condition: This section is used to specify a condition against the payload in order to determine whether an action needs to be executed or not:

"condition" : {
    "compare" : { "ctx.payload.hits.total" : { "gt" : 0 }}
  }

As shown in the preceding code, it uses a condition of the compare type to determine whether the payload has any documents, and if it finds any, then the action will be invoked. 

A condition of the compare type is used to specify simple comparisons, such as eq, not-eq, gt, gte, lt, and lte, against a value in the watch payload.

• actions: This section is used to specify one or more actions that need to be taken when the watch condition evaluates to true:

 "actions" : {
    "log_error" : {
      "logging" : {
        "text" : "The number of errors in logs is  {{ctx.payload.hits.total}}"
      }
    }
  }

As shown in the preceding code, it uses a logging action to log the specified text when the watch condition is met. The logs would be logged into Elasticsearch logs. The number of errors found is dynamically obtained using the field (hits.total) of the payload. The payload is accessed using the ctx.payload.* variable.

During the watches' execution, once the condition is met, a decision is made per configured action as to whether it should be throttled or continue executing the action. The main purpose of action throttling is to prevent too many executions of the same action for the same watch.

Watcher supports two types of throttling:

• Time-based Throttling: You can define a throttling period by using the throttle_period parameter as part of the action configuration or at the watch level (which applies to all actions) to limit how often the action is executed. The global default throttle period is 5 seconds.

• ACK-based Throttling: Using ACK Watch APIs, you can prevent watch actions from being executed again while the watch condition remains true.

Watches are stored in a special index named .watches. Every time a watch is executed, a watch_record containing details such as watch details, the time of watch execution, watch payload, and the result of the condition is stored in the watch history index, which is named .watches-history-6-*.