In the cloud world, we need to carry out monitoring to observe the progress and quality of our services and applications over a period of time. Monitoring allows us to keep our applications under systematic review. If something breaks, we want to know what it is and what caused it to malfunction. Monitoring helps us to investigate the failure points in our services. We can make sure that we detect these services early on using anomaly detection. White-box monitoring can help us work out which services are failing and why, and also how to debug them. It can also provide future trends, which means it can detect potential future failures. Here, we will be focusing only on tools that enable us to monitor either our application or our infrastructure:

• Monitoring the application: It is very important that the features or services that are being developed are monitored. There should be a proper time-series graph and a dashboard.
• Monitoring the infrastructure: If possible, all non-functional services should be monitored, with alerts, such as response times or requests per second. This is useful to catch small issues before they turn into large issues.

We need to monitor our servers and services to fix problems before the end users or customers come across them. Monitoring improves the product quality and makes sure customers are delighted with our services. It also helps us to avoid outage costs. If we have proper monitoring, we can make decisions based on data instead of hunches. In this chapter, we will not only look at how to set up monitoring, we will also enable alerts to notify us when systems are failing. We will then enable alerts to monitor the infrastructure to gather data on errors, performance, and throughput. After that, we will use alerts to measure the user experience of our application and the quality of the services we provide. This will allow us to make good decisions and measure the value we deliver.

In this chapter, we will discuss how we can monitor applications or services running on clusters, pods, and Kubernetes using Prometheus and Grafana. We will cover the following topics:

• Introducing Prometheus and the Prometheus architecture
• Setting up Prometheus
• Configuring alerts from Prometheus
• Introducing Grafana
• Setting up Grafana
• Configuring alerts from Grafana