In this chapter, we had the opportunity to experiment with a couple of useful tools to troubleshoot and analyze Prometheus configuration issues and performance. We started with promtool and went through all its available options; then, we used several endpoints and logs to ensure everything was working as expected. Finally, we described the tsdb tool and how it can be used to troubleshoot and pinpoint problems with cardinality and the churn of metrics and labels in our Prometheus database.

We can now step into recording and alerting rules, which will be covered in the next chapter.