Often, monitoring aims to collect technical metrics with a high frequency of many collections per second. The problem with this high frequency collection is that it heavily impacts the performance of the system. Metrics often get collected even if there is no performance regression.

As mentioned, application-level metrics, such as resource consumption, alone don't help much in identifying potential performance constraints. In the same way, the collection disrupts the responsiveness of the system.

Instead of monitoring with a high frequency it's advisable to sample metrics with lower frequency, such as for only a few times per minute. The theory behind statistical populations shows that these few samples represent the population of data well enough.

Sampling the information should have as little impact on the application's performance as possible. All subsequent investigations, or metrics querying or calculations should happen out-of-band; that is, outsourced to a system that does not impact the running application. The concerns for sampling the information from storing, querying, and displaying it, are thus separated.