Provide standardized traffic metrics

[tomkerkhove]

What would you like to be added:
Provide standardized traffic metrics for all gateways to implement so that other tools can rely on a common way to get the metrics.

Ideally, this would fully align with the metrics that SMI spec leverages to make it, even more, easier to integrate.

I've proposed to extend SMI to beyond Service Meshes but then this project started so it might be better to just align instead of re-invent.

Why is this needed:

Tools & platforms need a unified way to get traffic metrics for all gateways, regardless of what the effective gateway is that is being used.

In my case, we are building HTTP-based autoscaling for KEDA (experimental) so we will rely on things such as SMI, but here we were hoping to rely on a standard/SDK for getting the metrics as well instead of re-implementing every gateway.

/cc @michelleN @bridgetkromhout

[robscott]

Hey @tomkerkhove, thanks for bringing this up! We're always interested in finding any overlap with SMI Spec or any other related projects and trying to develop a standard that works for all use cases. In this case, metrics are something that we haven't prioritized yet, but something that could be in scope in the future. This actually came up in Slack a couple weeks ago and I raised it at our community meeting as well.

A few quick follow up questions related to SMI metrics:

1. Did you envision Gateway API supporting the same set of metrics? A subset/superset? Are there metrics that would be more relevant to only certain use cases?
2. What level of standardization do you think would be ideal to provide?
3. Would it be possible to expose this with standard prometheus metrics instead of a CRD?
4. In your experience, how difficult has it been for different implementations to provide this standardized set of metrics?
5. Are there any areas you'd like to improve on with the metrics in SMI?

[tomkerkhove]

Thanks for your reply!

In terms of what metrics I do not have much of a preference, but # of requests is the minimum for me.

A CRD is ideal for us/me as we do not want to force Prometheus on everyone. For example, some companies rely on their cloud provider for that so don't need it (including me).

In terms of standardization, I'd hope every gateway API has to/will provide these metrics and hopefully compatible with SMI so that we can have the same metric experience for Service Meshes, Ingresses/gateways and eventually hoping for service-to-service as well.

[beriberikix]

I would suggest success/failure vs. raw count, since success rate or # of requests can both be calculated from it.

[tomkerkhove]

Yes, but ideally they are there out of the box so no computation needs to be done which makes it easier to use

[bowei]

Given the SMI experience, are there any issues with using CRs (which are backed by etcd db) to store metrics that are fast changing and tend to be emphemeral? I know that K8S metrics resources have special handling to avoid overwhelming the API server.

(https://github.com/kubernetes/metrics/blob/master/pkg/apis/metrics/v1beta1/types.go)

[evankanderson]

Other art happening concurrently:

https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/metrics/semantic_conventions/http-metrics.md

I'd share bowei's concern about the apiserver not necessarily being the best place for fast-moving metrics.

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-contributor-experience at kubernetes/community.
/lifecycle stale

[tomkerkhove]

Closing as discussed in standup that this is not on the radar for now.

[tomkerkhove]

Since time has passed by, I'm wondering if there are plans to integrate Traffic Metrics into the spec? If so, would be nice to re-use some of the SMI spec or OpenTelemetry ones.

[howardjohn]

I am admittedly not very familiar with them, but my understanding is that opentelemetry also defined some standardized request metric schemas which could be a reasonable API if this project decides to 'endorse' a metrics scheme.

[tomkerkhove]

That's definitely correct, but what if it's not endorsed and rather part of the spec to provide these?

This is what SMI does and allows end-users to rely on a standard way of getting metrics; regardless of what standard that is being used for the semantics. Tooling knows they are there, for every Gateway API.