Skip to content

Commit c5db3a5

Browse files
vaxvmsvaxvms
committed
Add a prometheus label mapping component
1 parent 48f9206 commit c5db3a5

File tree

7 files changed

+487
-0
lines changed

7 files changed

+487
-0
lines changed

‎CHANGELOG.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -22,6 +22,8 @@ Main (unreleased)
2222

2323
- Add `otelcol.exporter.syslog` component to export logs in syslog format (@dehaansa)
2424

25+
- Add a `prometheus.mapping` component to add labels based on a source_label and a mapping table. (@vaxvms)
26+
2527
### Enhancements
2628

2729
- Add second metrics sample to the support bundle to provide delta information (@dehaansa)

‎docs/design/2025-prometheus-mapping-component.md

Lines changed: 45 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,45 @@
1+
# Proposal: Add a component to perform label mapping efficiently
2+
3+
* Author(s): Nicolas DUPEUX
4+
* Last updated: 19/11/2024
5+
* Original issue: https://github.com/grafana/alloy/pull/2025
6+
7+
## Abstract
8+
9+
Add a component to populate labels values based on a lookup table.
10+
11+
## Problem
12+
13+
Using `prometheus.relabel` to populate a label value based on another label value is inefficient as we have to have a rule block for each source label value.
14+
15+
If we have 1k values to map, we'll have to execute 1k regex for each datapoint resulting in an algorithm complexity of O(n).
16+
17+
## Proposal
18+
19+
Replace regex computing by a lookup table. Algorithm complexity goes from O(n) to O(1)
20+
21+
## Pros and cons
22+
23+
Pros:
24+
- resource efficient
25+
26+
Cons:
27+
- New component
28+
29+
## Alternative solutions
30+
31+
- Instanciate more CPU resources to perform the task
32+
- Optimize prometheus.relabel component
33+
- Summarize regex when severals keys have to same value.
34+
35+
## Compatibility
36+
37+
As this is a new component, there isn't any compatibility issue as long as you don't use it.
38+
39+
## Implementation
40+
41+
https://github.com/grafana/alloy/pull/2025
42+
43+
## Related open issues
44+
45+
None

‎docs/sources/reference/compatibility/_index.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -173,6 +173,7 @@ The following components, grouped by namespace, _export_ Prometheus `MetricsRece
173173
{{< /collapse >}}
174174

175175
{{< collapse title="prometheus" >}}
176+
- [prometheus.mapping](../components/prometheus/prometheus.mapping)
176177
- [prometheus.relabel](../components/prometheus/prometheus.relabel)
177178
- [prometheus.remote_write](../components/prometheus/prometheus.remote_write)
178179
- [prometheus.write.queue](../components/prometheus/prometheus.write.queue)
@@ -192,6 +193,7 @@ The following components, grouped by namespace, _consume_ Prometheus `MetricsRec
192193
{{< /collapse >}}
193194

194195
{{< collapse title="prometheus" >}}
196+
- [prometheus.mapping](../components/prometheus/prometheus.mapping)
195197
- [prometheus.operator.podmonitors](../components/prometheus/prometheus.operator.podmonitors)
196198
- [prometheus.operator.probes](../components/prometheus/prometheus.operator.probes)
197199
- [prometheus.operator.servicemonitors](../components/prometheus/prometheus.operator.servicemonitors)

‎docs/sources/reference/components/prometheus/prometheus.mapping.md

Lines changed: 132 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,132 @@
1+
---
2+
canonical: https://grafana.com/docs/alloy/latest/reference/components/prometheus/prometheus.mapping/
3+
description: Learn about prometheus.mapping
4+
title: prometheus.mapping
5+
---
6+
7+
# prometheus.mapping
8+
9+
Prometheus metrics follow the [OpenMetrics](https://openmetrics.io/) format.
10+
Each time series is uniquely identified by its metric name, plus optional
11+
key-value pairs called labels. Each sample represents a datapoint in the
12+
time series and contains a value and an optional timestamp.
13+
14+
```text
15+
<metric name>{<label_1>=<label_val_1>, <label_2>=<label_val_2> ...} <value> [timestamp]
16+
```
17+
18+
The `prometheus.mapping` component create new labels on each metric passed
19+
along to the exported receiver by applying a mapping table to a label value.
20+
21+
The most common use of `prometheus.mapping` is to create new labels with a high
22+
cardinality source label value (>1k) when a large set of regex is inefficient.
23+
24+
You can specify multiple `prometheus.mapping` components by givin them
25+
different labels.
26+
27+
## Usage
28+
29+
```alloy
30+
prometheus.mapping "LABEL" {
31+
forward_to = RECEIVER_LIST
32+
33+
source_label = "labelA"
34+
35+
mapping = {
36+
"from" = {"labelB" = "to"},
37+
...
38+
}
39+
}
40+
```
41+
42+
## Arguments
43+
44+
The following arguments are supported:
45+
46+
Name | Type | Description | Default | Required
47+
---------------|---------------------------|-------------------------------------------------------------------------|---------|---------
48+
`forward_to` | `list(MetricsReceiver)` | Where the metrics should be forwarded to, after relabeling takes place. | | yes
49+
`source_label` | `string` | Name of the source label to use for mapping. | | yes
50+
`mapping` | `map(string,map(string))` | Mapping from source label value to target labels name/value. | | yes
51+
52+
## Exported fields
53+
54+
The following fields are exported and can be referenced by other components:
55+
56+
Name | Type | Description
57+
-----------|-------------------|-----------------------------------------------------------
58+
`receiver` | `MetricsReceiver` | The input receiver where samples are sent to be relabeled.
59+
60+
## Component health
61+
62+
`prometheus.mapping` is only reported as unhealthy if given an invalid
63+
configuration. In those cases, exported fields are kept at their last healthy
64+
values.
65+
66+
## Debug information
67+
68+
`prometheus.mapping` doesn't expose any component-specific debug information.
69+
70+
## Debug metrics
71+
72+
* `prometheus_mapping_metrics_processed` (counter): Total number of metrics processed.
73+
* `prometheus_mapping_metrics_written` (counter): Total number of metrics written.
74+
75+
## Example
76+
77+
You can create and an instance of a see `prometheus.mapping` component and see how
78+
it acts on the following metrics.
79+
80+
```alloy
81+
prometheus.mapping "keep_backend_only" {
82+
forward_to = [prometheus.remote_write.onprem.receiver]
83+
84+
source_label = "app"
85+
86+
mapping = {
87+
"frontend" = {"team" = "teamA"}
88+
"backend" = {"team" = "teamB"}
89+
"database" = {"team" = "teamC"}
90+
}
91+
}
92+
```
93+
94+
```
95+
metric_a{__address__ = "localhost", instance = "development", app = "frontend"} 10
96+
metric_a{__address__ = "localhost", instance = "development", app = "backend"} 2
97+
metric_a{__address__ = "cluster_a", instance = "production", app = "frontend"} 7
98+
metric_a{__address__ = "cluster_a", instance = "production", app = "backend"} 9
99+
metric_a{__address__ = "cluster_b", instance = "production", app = "database"} 4
100+
```
101+
102+
After applying the mapping a new `team` label is created based on mapping table
103+
and `app` label value.
104+
105+
```
106+
metric_a{team = "teamA", __address__ = "localhost", instance = "development", app = "frontend"} 10
107+
metric_a{team = "teamB", __address__ = "localhost", instance = "development", app = "backend"} 2
108+
metric_a{team = "teamA", __address__ = "cluster_a", instance = "production", app = "frontend"} 7
109+
metric_a{team = "teamA", __address__ = "cluster_a", instance = "production", app = "backend"} 9
110+
metric_a{team = "teamC", __address__ = "cluster_a", instance = "production", app = "database"} 4
111+
```
112+
113+
The resulting metrics are then propagated to each receiver defined in the
114+
`forward_to` argument.
115+
<!-- START GENERATED COMPATIBLE COMPONENTS -->
116+
117+
## Compatible components
118+
119+
`prometheus.mapping` can accept arguments from the following components:
120+
121+
- Components that export [Prometheus `MetricsReceiver`](../../../compatibility/#prometheus-metricsreceiver-exporters)
122+
123+
`prometheus.mapping` has exports that can be consumed by the following components:
124+
125+
- Components that consume [Prometheus `MetricsReceiver`](../../../compatibility/#prometheus-metricsreceiver-consumers)
126+
127+
{{< admonition type="note" >}}
128+
Connecting some components may not be sensible or components may require further configuration to make the connection work correctly.
129+
Refer to the linked documentation for more details.
130+
{{< /admonition >}}
131+
132+
<!-- END GENERATED COMPATIBLE COMPONENTS -->

‎internal/component/all/all.go

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -132,6 +132,7 @@ import (
132132
_ "github.com/grafana/alloy/internal/component/prometheus/exporter/statsd" // Import prometheus.exporter.statsd
133133
_ "github.com/grafana/alloy/internal/component/prometheus/exporter/unix" // Import prometheus.exporter.unix
134134
_ "github.com/grafana/alloy/internal/component/prometheus/exporter/windows" // Import prometheus.exporter.windows
135+
_ "github.com/grafana/alloy/internal/component/prometheus/mapping" // Import prometheus.mapping
135136
_ "github.com/grafana/alloy/internal/component/prometheus/operator/podmonitors" // Import prometheus.operator.podmonitors
136137
_ "github.com/grafana/alloy/internal/component/prometheus/operator/probes" // Import prometheus.operator.probes
137138
_ "github.com/grafana/alloy/internal/component/prometheus/operator/servicemonitors" // Import prometheus.operator.servicemonitors

0 commit comments

Comments
 (0)