-
Notifications
You must be signed in to change notification settings - Fork 881
PrometheusExporterMiddleware fails if no metrics have run yet #2970
Description
Bug Report
List of all OpenTelemetry NuGet packages and version that you are using (e.g. OpenTelemetry 1.0.2):
- OpenTelemetry 1.2.0-rc2
- OpenTelemetry.Exporter.Prometheus 1.2.0-rc2
Runtime version (e.g. net461, net48, netcoreapp3.1, net5.0 etc. You can find this information from the *.csproj file):
- net6.0
Symptom
Metrics scraping yields a 500 if no metrics have been generated yet. This is primarily a problem in early app execution where there may be counters defined but none have been incremented yet.
What is the expected behavior?
I expect the middleware to yield a 200 response with no statistics in the body.
What is the actual behavior?
There's a 500 response body where the error message is "Collection failure." This is actually really hard to troubleshoot because, depending on the setup that message may or may not be logged.
Reproduce
using System.Diagnostics.Metrics;
using Microsoft.AspNetCore.Builder;
using Microsoft.AspNetCore.Http;
using Microsoft.Extensions.DependencyInjection;
using OpenTelemetry.Metrics;
namespace Reproduction;
public class ReproFixture
{
[Fact]
public async Task ReproTest()
{
var services = new ServiceCollection()
.AddOpenTelemetryMetrics(b =>
{
b.AddMeter("test-meter");
b.AddPrometheusExporter(opt => opt.StartHttpListener = false);
})
.BuildServiceProvider();
var builder = new ApplicationBuilder(services);
builder.UseOpenTelemetryPrometheusScrapingEndpoint();
var app = builder.Build();
var meter = new Meter("test-meter");
var counter = meter.CreateCounter<long>("counter");
// Uncomment this to make the test pass.
// counter.Add(5);
var context = new DefaultHttpContext();
context.Request.Path = "/metrics";
await app.Invoke(context);
// Response will be 500 if the counter has not been incremented.
Assert.Equal(200, context.Response.StatusCode);
}
}Additional Context
This happens because the middleware throws if the collection results are empty. I think the answer to this is to let the collection results be empty. Optionally, it may be worth logging a warning (e.g., using an ILogger) if there aren't any metrics and provide some information about that.