The prometheus-net NuGet package provides a set of default metric collectors and a MetricServer class that provides the instrumentation endpoint that Prometheus hooks into. This package is great for adding Prometheus support to any app. The metrics are provided by a self-hosted HTTP endpoint, and you can provide custom metrics for your application.

In the dockeronwindows/ch11-api-with-metrics image, I've added Prometheus support into a Web API project. The code to configure and start the metrics endpoint is in the PrometheusServer class:

public static void Start()
{
  _Server = new MetricServer(50505);
  _Server.Start();
}

This starts a new MetricServer instance listening on port 50505 and running the default set of .NET statistics and performance counter collectors that the NuGet package provides. These are on-demand collectors, which means they provide metrics when the Prometheus server calls into the endpoint.

The MetricServer class will also return any custom metrics you set up in your application. Prometheus supports different types of metrics. The simplest is the counter, which is just an incrementing counter—Prometheus queries your app for the metrics values, and the app returns a single number for each counter. In the ValuesController class, I have set up some counters to record requests and responses to the API:

private Counter _requestCounter = Metrics.CreateCounter("ValuesController_Requests", 
                                                         "Request count", "method", "url");

private Counter _responseCounter = Metrics.CreateCounter("ValuesController_Responses", 
                                                         "Response count", "code", "url");

When requests come into the controller, the controller action method increments the request count for the URL and increments the status count for the response code by calling the Inc() method on the counter objects:

public IHttpActionResult Get()
{
  _requestCounter.Labels("GET", "/").Inc();
  _responseCounter.Labels("200", "/").Inc();
  return Ok(new string[] { "value1", "value2" });
}

Prometheus has various other types of metrics that you can use to record key information about your app—counters only increase, but gauges can increase and decrease, so they're useful for recording snapshots. Prometheus records each metric value with a timestamp and a set of arbitrary labels that you provide. In this case, I will add the URL and the HTTP method to the request count and the URL and status code to the response count. I can use these to aggregate or filter metrics in Prometheus.

The counters I set up in the Web API controller give me a set of custom metrics showing which endpoints are being used and the status of the responses. These are exposed by the server component in the NuGet package, along with the default metrics to record the system's performance. In the Dockerfile for this app, there are two additional lines for the Prometheus endpoint:

EXPOSE 50505
RUN netsh http add urlacl url=http://+:50505/metrics user=BUILTINIIS_IUSRS; `
    net localgroup 'Performance Monitor Users' 'IIS APPPOOLDefaultAppPool' /add

The first line just exposes the custom port I'm using for the metrics endpoint. The second line sets up the permissions that are needed for that endpoint. In this case, the metrics endpoint is hosted inside the ASP.NET app, so the IIS user account needs permissions to listen on the custom port and to access the system performance counters.

You can build the Dockerfile and run a container from the image in the usual way, that is, by publishing all the ports with -P:

docker container run -d -P --name api dockeronwindows/ch11-api-with-metrics:2e

To check whether the metrics are being recorded and exposed, I can run some PowerShell commands to grab the port of the container, then make some calls to the API endpoint and check the metrics:

$apiPort = $(docker container port api 80).Split(':')[1]
for ($i=0; $i -lt 10; $i++) {
 iwr -useb "http://localhost:$apiPort/api/values"
}

$metricsPort = $(docker container port api 50505).Split(':')[1]
(iwr -useb "http://localhost:$metricsPort/metrics").Content

You'll see a plain text list of metrics, grouped by name and label. Each metric also contains the metadata for Prometheus, including the metric name, the type, and a friendly description:

# HELP process_num_threads Total number of threads
# TYPE process_num_threads gauge
process_num_threads 27
# HELP dotnet_total_memory_bytes Total known allocated memory
# TYPE dotnet_total_memory_bytes gauge
dotnet_total_memory_bytes 8519592
# HELP process_virtual_memory_bytes Virtual memory size in bytes.
# TYPE process_virtual_memory_bytes gauge
process_virtual_memory_bytes 2212962820096
# HELP process_cpu_seconds_total Total user and system CPU time spent in seconds.
# TYPE process_cpu_seconds_total counter
process_cpu_seconds_total 1.734375
...
# HELP ValuesController_Requests Request count
# TYPE ValuesController_Requests counter
ValuesController_Requests{method="GET",url="/"} 10
# HELP ValuesController_Responses Response count
# TYPE ValuesController_Responses counter
ValuesController_Responses{code="200",url="/"} 10

The complete output is much larger. In this snippet, I've shown the total number of threads, the allocated memory and the CPU usage, which all comes from standard Windows and .NET performance counters inside the container. I've also shown the custom HTTP request and response counters.

My custom counters in this application show the URL and the response code. In this case, I can see 10 requests to the root URL of the value controller, and ten responses with the OK status code 200. Later in this chapter, I'll show you how to visualize these statistics using Grafana.

Adding the NuGet package to the project and running the MetricServer is a simple extension of the source code. It lets me record any kind of metric that is useful, but it does mean changing the app, so it's only suitable for applications that are under active development.

In some cases, you may want to add monitoring without altering the application you want to instrument. In that case, you can run an exporter alongside your app. An exporter pulls metrics from your application process and exposes them to Prometheus. In Windows containers, you can get a lot of useful information from the standard performance counters.