metrics enrich component

[ArkovKonstantin]

PR Description

This PR contains an implementation of the component prometheus.enrich described in this proposal

Notes to the Reviewer

The integration test demonstrates the enrichment of metrics collected from the prom-gen service using file-based discovery sample

PR Checklist

• CHANGELOG.md updated
• Documentation added
• Tests updated
• Config converters updated

[CLAassistant]

All committers have signed the CLA.

[clayton-cornell]

Generally, the Usage section doesn't go into depth with explanations in comments. This section usually just details the high level structure for the component configuration.

[clayton-cornell]

Docs look Ok for now. Now it's over to the dev team for a code review.

[ptodev]

Hi, @ArkovKonstantin, thank you for the PR! Have you tried using array.combine_maps? We introduced it to solve similar issues in a more general way.

[ArkovKonstantin]

Hi, @ptodev

Yes, I tried using the array.combine_maps function, but I couldn't apply it to the scenario where I needed to enrich metrics coming from the prometheus.scrape or prometheus.receive_http components. The reason is that both of these components accept inputs from components that export a MetricReceiver, whereas array.combine_maps takes two arguments of type list(map(string)) and cannot operate within the metric flow.

Because of this limitation, I decided to propose an implementation of the prometheus.enrich component, which exports a MetricReceiver so that it can be used directly with prometheus.scrape, prometheus.receive_http or prometheus.relabel. At the same time, it accepts targets of type list(map(string)) as input, allowing it to receive enrichment data from discovery components.

Below is an examples of how to use the prometheus.enrich component

// Discover device metadata from file
discovery.file "network_devices" {  
   files = ["/etc/alloy/devices.json"]  
}  

// Collects metrics of prom-gen:9001
prometheus.scrape "default" {  
  targets = [  
    {"__address__" = "prom-gen:9001"},  
  ]  
  
  forward_to = [prometheus.enrich.default.receiver]  
}  
  
// Enrich metrics with device metadata  
prometheus.enrich "default" {  
    targets = discovery.file.network_devices.targets  
  
    target_match_label = "hostname"  
    metrics_match_label = "instance"  
  
    labels_to_copy = ["datacenter", "environment"]  
  
    forward_to = [prometheus.remote_write.default.receiver]  
}

// Write metrics to mimir
prometheus.remote_write "default" {  
  endpoint {  
    url = "http://mimir:9009/api/v1/push"    
  }    
}

// Discover device metadata from file
discovery.file "network_devices" {  
   files = ["/etc/alloy/devices.json"]  
}  
  

// Receives metrics over HTTP
prometheus.receive_http "default" {
  http {
    listen_address = "0.0.0.0"
    listen_port = 9999
  }
  forward_to = [prometheus.enrich.default.receiver]
}

// Enrich metrics with device metadata
prometheus.enrich "default" {  
    targets = discovery.file.network_devices.targets  
  
    target_match_label = "hostname"  
    metrics_match_label = "instance"  
  
    labels_to_copy = ["datacenter", "environment"]  
  
    forward_to = [prometheus.remote_write.default.receiver]  
}

// Write metrics to mimir
prometheus.remote_write "default" {  
  endpoint {  
    url = "http://mimir:9009/api/v1/push"    
  }    
}

[ptodev]

Thanks, LGTM! I just added a few minor comments. I'll speak with other colleagues about the PR today, to give them a heads up, but I'm personally happy with it.

[ptodev]

Suggested change

	metrics_match_label = "<LABEL>"
	labels_to_copy = ["<LABEL>", ...]

The Usage section is for minimal usage. It only lists mandatory arguments and blocks.

[ptodev]

Would you mind also adding the prometheus.receive_http example from this comment please?

[ptodev]

It'd be nice to mention that if a match doesn't happen, the metrics will be passed through.

[ptodev]

Suggested change

	* `prometheus_target_cache_size` (gauge): Total size of targets cache.
	* `prometheus_target_cache_size` (gauge): The number of targets in the `targets` list.

[ptodev]

@ArkovKonstantin btw another issue with array.combine_maps is that it currently only does an inner join. If there's no match it will not output any targets. I think it should be updated with an extra argument so that it can output something even if there's no match... this would probably be the way most people would want to use it. It should be more performant than prometheus.enrich in situations where the enrichment could be done prior to the scrape, but I agree that prometheus.enrich will be a good solution for other situations.