Not all application errors will result in an error response — there are times when the application logic has a bug that results in a successful response but with the wrong values. In those cases, you need to have some way to correlate between the response the user received and the log events in your system. This is the purpose of the correlation identifier (or CorrelationID). 

The CorrelationID is a value that you create when a request is received, which you use whenever you write a log event. When the request handling is done, you attach the CorrelationID to the response, usually as a header, to allow the user to communicate it back to you if needed. You can even enhance this technique and send the CorrelationID to other services your API consumes, so later, when investigating an issue, you can rebuild the entire flow that the logical transaction went through.

Application Insights already generates a unique Operation ID for each request, so we can leverage this value and send it as a header in the response. The typical header name for a correlation identifier is X-Correlation-ID.

To add a header for each response, we need to intercept the request pipeline and implement some middleware that will inject the header when the response is being written.

Here is a simple implementation of such a piece of middleware:

public class CorrelationIdHeaderMiddleware
{
    private const string CorrelationHeaderKey = "X-Correlation-ID";

    private readonly RequestDelegate _next;

    public CorrelationIdHeaderMiddleware(RequestDelegate next)
    {
        _next = next;
    }

    public async Task Invoke(HttpContext context)
    {
        // Retrieve the current Application Insight Telemetry object for the request
        var requestTelemetry = context.Features.Get<RequestTelemetry>();

        // Register to be notified when the headers are written to the response
        context.Response.OnStarting(_ =>
        {
            // Add the Correlation ID header when the response is being written 
            context.Response.Headers.Add(CorrelationHeaderKey, new[] { requestTelemetry.Id });
            return Task.CompletedTask;
        }, null);

        // Continue the execution pipeline
        await _next(context);
    }
}

With the implementation of the middleware ready, you need to add it to your application. To make it easier and more readable, I created an extension method that sets the middleware to the ASP.NET Application Builder:

public static class CorrelationIdMiddlewareExtensions
{
    public static void UseCorrelationIdHeader(this IApplicationBuilder app)
    {
        app.UseMiddleware<CorrelationIdHeaderMiddleware>();
    }
}

Finally, you need to use the extension method when configuring the application in the Startup class:

public class Startup
{       
    ...
   
    public void Configure(IApplicationBuilder app, IHostingEnvironment env)
    {
        ...
       
        // Each response will now include a 'X-Correlation-ID' header 
        app.UseCorrelationIdHeader();

        ...
    }
}

Now, whenever you make a request to the API, the response will include the X-Correlation-ID header that holds the same value as the Operation ID that Application Insights uses when writing the logging events. 

For example, here is the response I received after using the search API without providing a category name:

Having the CorrelationID allows me to go through the logs I have in my Application Insights Search and see what happened with this request. For brevity, I filtered the messages that are marked Debug level: