Inside this chapter's test environment, we can find the following Prometheus configuration:
vagrant@prometheus:~$ cat /etc/prometheus/prometheus.yml
global:
...
evaluation_interval: 1m
...
rule_files:
- "recording_rules.yml"
- "alerting_rules.yml"
alerting:
alertmanagers:
- static_configs:
- targets:
- “prometheus:5001”
...
In this configuration, there are three components to be aware of:
- evaluation_interval: This is responsible for defining the global evaluation interval for recording and alerting rules, which can be overridden at the rule group level using the interval keyword.
- rule_files: This is the file location where Prometheus can read the configured recording and/or alerting rules.
- alerting: This is the endpoint(s) where Prometheus sends alerts for further processing.
In the alerting section, we've configured “prometheus:5001”. Behind this endpoint, there is nothing more than a small service, called alertdump, that is listening on port 5001 for the HTTP POST requests and is simply dumping their payload onto a log file. This will help dissect what Prometheus sends when an alert is firing.