Prometheus is an open source monitoring and alerting tool. It gathers time series-based numerical data from the applications being monitored. Written in the GO language, the tool captures metrics in the metrics 2.0 format (http://metrics20.org/): the metrics have a name, a description, dimensions, and values. The only thing missing is a unit for the metrics.

Prometheus is used for Whitebox monitoring, where, the application being monitored is aware that it is being monitored. Endpoint-defining metrics are exposed over HTTP by the application. Prometheus uses various exporters that can share data with the server. One of the most widely used exporters is NodeExporter. When NodeExporter is run on a host, it will provide details on I/O, memory, disk, and CPU pressure. You can create exporters to monitor almost anything in the service, such as API calls, method calls, or database interactions.

The tool provides PromQL, a sophisticated query language to fetch time series data is stored.

The following diagram shows the high-level architecture of Prometheus:

Image Source :
https://prometheus.io/docs/introduction/overview/#architecture

The architecture showcases the following components:

•  Prometheus server: This server collects the metrics from applications and stores them locally. The Prometheus server works on the principle of scraping, that is, invoking the metrics endpoints of the various nodes that it is configured to monitor. It collects these metrics at regular intervals and stores them locally.
• Push gateway: There are the cases when an endpoint cannot be exposed by the application due to nature of its work, such as static jobs. The Push gateway captures the data, transforms that data into the Prometheus data format, and then pushes that data onto the Prometheus server.
• Alert manager: Can deliver alerts to multiple channels, such as SMS, email, and Slack, based on alerting rules.