Skip to content

PrometheusExporterMiddleware fails if no metrics have run yet #2970

@tillig

Description

@tillig

Bug Report

List of all OpenTelemetry NuGet packages and version that you are using (e.g. OpenTelemetry 1.0.2):

  • OpenTelemetry 1.2.0-rc2
  • OpenTelemetry.Exporter.Prometheus 1.2.0-rc2

Runtime version (e.g. net461, net48, netcoreapp3.1, net5.0 etc. You can find this information from the *.csproj file):

  • net6.0

Symptom

Metrics scraping yields a 500 if no metrics have been generated yet. This is primarily a problem in early app execution where there may be counters defined but none have been incremented yet.

What is the expected behavior?

I expect the middleware to yield a 200 response with no statistics in the body.

What is the actual behavior?

There's a 500 response body where the error message is "Collection failure." This is actually really hard to troubleshoot because, depending on the setup that message may or may not be logged.

Reproduce

using System.Diagnostics.Metrics;
using Microsoft.AspNetCore.Builder;
using Microsoft.AspNetCore.Http;
using Microsoft.Extensions.DependencyInjection;
using OpenTelemetry.Metrics;

namespace Reproduction;

public class ReproFixture
{
    [Fact]
    public async Task ReproTest()
    {
        var services = new ServiceCollection()
            .AddOpenTelemetryMetrics(b =>
                {
                    b.AddMeter("test-meter");
                    b.AddPrometheusExporter(opt => opt.StartHttpListener = false);
                })
            .BuildServiceProvider();
        var builder = new ApplicationBuilder(services);
        builder.UseOpenTelemetryPrometheusScrapingEndpoint();
        var app = builder.Build();
        var meter = new Meter("test-meter");
        var counter = meter.CreateCounter<long>("counter");

        // Uncomment this to make the test pass.
        // counter.Add(5);

        var context = new DefaultHttpContext();
        context.Request.Path = "/metrics";
        await app.Invoke(context);

        // Response will be 500 if the counter has not been incremented.
        Assert.Equal(200, context.Response.StatusCode);
    }
}

Additional Context

This happens because the middleware throws if the collection results are empty. I think the answer to this is to let the collection results be empty. Optionally, it may be worth logging a warning (e.g., using an ILogger) if there aren't any metrics and provide some information about that.

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions