dataflow task metrics "task_external_execution_id" label doesn't have value

[guoyiang]

Description:
I deployed spring cloud dataflow in kubernetes using helm chart and enabled prometheus metrics.

I noticed some labels are not set with correct values, as you can see in the metrics collected in spring-cloud-dataflow-prometheus-proxy:8080/metrics/connected:

jvm_buffer_total_capacity_bytes{application="unknown-unknown",id="mapped",service="task-application",task_execution_id="234",task_external_execution_id="unknown",task_name=" test-tasks",task_parent_execution_id="unknown",} 0.0

application and task_external_execution_id are both unknown.

Release versions:
helm chart version: 2.5.2
dataflow server version: 2.7.0

Additional context:

From execution history, the below parameter are passed into task execution.

--management.metrics.tags.service: task-application
--management.metrics.export.prometheus.enabled: true
--spring.datasource.url: ******
--spring.datasource.driverClassName: org.postgresql.Driver
--management.metrics.tags.application: ${task.name:unknown}-${task.execution.id:unknown}
--management.metrics.export.prometheus.rsocket.port: 7001
--management.metrics.export.prometheus.rsocket.enabled: true
--spring.cloud.task.name: test-tasks
--management.metrics.export.prometheus.rsocket.host: spring-cloud-dataflow-prometheus-proxy
--spring.cloud.task.executionid: 234

application tag refer to ${task.name:unknown} and ${task.execution.id:unknown}, but the actual parameters are spring.cloud.task.name and spring.cloud.task.executionid. This is reported in #4299 .

task_external_execution_id is not provided in the arguments and task was not able to set a meaningful value either. It's supposed to be the pod name as we see in dashboard.

A workaround to fix application tag is setting task.name and task.execution.id from spring.cloud.task.name and spring.cloud.task.execution.id in task's application.yaml config, but I haven't find a workaround for task_external_execution_id yet.

Thanks in advance!

[tzolov]

@guoyiang, thank you for highlighting this issue!
The default task metrics tags are configured here: https://github.com/spring-cloud/spring-cloud-dataflow/blob/master/spring-cloud-dataflow-server-core/src/main/resources/META-INF/application-task-common-properties-defaults.yml

Apparently from the initial commit they've been incorrect?
@cppwfs if you can confirm that the spring.cloud.task.name and spring.cloud.task.executionid are the correct I will submit a PR!
I would also add a task+metrics IT tests (similar to analyticsCounter() stream metrics test) to have end to end testing for this behavior.

[cppwfs]

@tzolov Thank you for looking into this. Yes those values are correct.

[guoyiang]

Very appreciate for the quick fix!

However, it doesn't seem to add external_execution_id though, will this info be added?