[bug] Prometheus Input -> Prometheus Client Output Using Namedrop: Metric Types Are Lost, Metrics Have Mangled Names With E.g _counter / _gauge Suffixes

[victorkamenov]

Relevant telegraf.conf:

###############################################################################
#                            OUTPUT PLUGINS                                   #
###############################################################################

# Configuration for the Prometheus client to spawn
[[outputs.prometheus_client]]
  namedrop = ["blah_*"]

  ## Address to listen on
  listen = ":9001"
  path = "/outputs/prometheus"

  ## Interval to expire metrics and not deliver to prometheus, 0 == no expiration
  expiration_interval = "$EXPIRE_INTERVAL"

  ## Collectors to enable, valid entries are "gocollector" and "process".
  ## If unset, both are enabled.
  collectors_exclude = ["gocollector", "process"]
	

###############################################################################
#                            INPUT PLUGINS                                    #
###############################################################################
[[inputs.prometheus]]
  ## An array of urls to scrape metrics from.
  urls = ["http://localhost:9091/metrics"]

System info:

Telegraf v1.7.2 (git: release-1.7 c6feb05)
Windows 10

Steps to reproduce:

1. Have the scrape target configured in the Prometheus Input Plugin URL conf ready (I used out of the box Prometheus Pushgateway, any other Prometheus target should do).
2. Run telegraf.exe -config newconf.txt

Expected behavior:

At the prometheus_client output path, I should see metrics similar to what I get on the Pushgateway's /metrics default telemetry endpoint.

Attaching a Pushgateway output sample:
pushgateway.txt

Actual behavior:

Instead, I see that the metric types are lost - please compare the metrics from the Pushgateway sample above (pushgateway.txt) and Telegraf (prometheus_client.txt) below:

prometheus_client.txt

The loss of metric types leads to _counter and _gauge suffixes being added to metric names , and for summaries and histograms - names with the quantiles in them, it seems, and without the related tags.

Additional info:

If I remove the
namedrop = ["blah_*"]

line in the prometheus_output client configuration, it works as expected, and the output produced by prometheus_client is without _gauge and _counter suffixes:

prometheus_client_without_namedrop.txt

I suspect the problem is in

internal/models/running_output.go , line 113 :
m, _ = metric.New(name, tags, fields, t)

I guess the metric type is lost there, when the metric is copied.

Changing the line to:
m, _ = metric.New(name, tags, fields, t, m.Type())

seems to fix the problem - metric types are no longer lost, metric names are not mangled with _counter and _gauge suffixes, but I don't know if this fix introduces other problems.

Thank you.

any updates?

[glinton]

I believe the fix has been released in Telegraf 1.7.3