Prometheus exporter produces zero-byte HTTP response

[Abrynos]

Package

OpenTelemetry.Exporter.Prometheus.AspNetCore

Package Version

Package Name	Version
OpenTelemetry.Exporter.Prometheus.AspNetCore	1.9.0-alpha.1
OpenTelemetry.Extensions.Hosting	1.9.0-alpha.1
OpenTelemetry.Instrumentation.AspNetCore	1.8.1
OpenTelemetry.Instrumentation.Http	1.8.1
OpenTelemetry.Instrumentation.Runtime	1.8.1

Runtime Version

net8.0

Description

Using Prometheus exporter 1.9.0-alpha.1 I'm getting a bunch of zero-byte responses from the metrics endpoint; status code is 200 and Content-Length header is 0.

This issue has been verified by another user as well:

It does not make a difference whether I access directly/behind nginx, via http/https, via hostname/IP-address or even with/without password configured in the application.

Here are corresponding TRACE logs (I've replaced my actual password of course 😉):

Steps to Reproduce

Two possibilities:

• Copy the example source code from the sources linked below into separate project.

  • Asp.Net startup (plugin is loaded at lines 249 and 404)
  • Plugin (lines 101 to 129)

• Set up ArchiSteamFarm in "generic" configuration and install it's "monitoring plugin". The application release is still tracking version 1.7.0-rc.1 so you either need to manually build or grab it from the build pipeline where we originally wanted to upgrade to the newest version over here.

Until now I was not yet able to reproduce it using only localhost but only on a remote host so keep in mind that you might need to have a phone or raspberry pi at hand.

Expected Result

Response from prometheus metrics endpoint contains metrics.

Actual Result

Response status code is 200 and Content-Length header is 0.

Additional Context

Thank you in advance for looking into this. If I can be of any assistance, please let me know.

[JustArchi]

It's important to add that version 1.7.0-rc.1 works properly. It's not the last version where it did work properly, however, we skipped higher ones due to another bug (#5506) which was fixed in 1.9.0.

To give you some additional context, we're originally using your project in https://github.com/JustArchiNET/ArchiSteamFarm. The setup is a bit more complex than usual, since opentelemetry library we've created integrates with the main project as optional addition - a plugin.

There is a non-zero chance that we're doing something wrong, but in this case, we'd appreciate a helpful hand, since it did work properly in version 1.7.0, and it no longer does in 1.9.0, without any change from us other than bumping the version, and without indicator in release notes or changelog that some core concept has changed which we should adapt to, thanks 🙂

[miniduikboot]

We're seeing similar issues after around 12 to 18 hours of uptime, although this depends on server load, with less loaded servers surviving for longer. We're also using a plugin to hook into a larger (ASP).Net application in a similar way as JustArchi.

[Abrynos]

less loaded servers surviving for longer

While this might correlate with the issue, I can't confirm it on my end. My system (Raspberry Pi 4 - 8GB) is running at basically no system load (see screenshot for the last 20 days) and I'm running into this within a few seconds of starting the application each time.

[raphaelquati]

Here, the same error occurs too. The metrics endpoint suddenly stop responding.

I reverted to 1.7.0-rc.1, and it's working again

[tai-yi]

👁️Sice 1.8.x OpenTelemetry.Exporter.Prometheus.AspNetCore became...
Every new version tried then have to revert to 1.7.0-rc.1(●'◡'●)