Skip to content

Nomad job alerts should use rate #477

@Lusitaniae

Description

@Lusitaniae

While nomad reports the metrics as a gauge, it actually behaves as a counter as far as I've seen

So the alerts should be doing rate() on the metrics

curl localhost:4646/v1/metrics?format=prometheus -s  | grep nomad_job | grep "#"
# HELP nomad_nomad_job_status_dead nomad_nomad_job_status_dead
# TYPE nomad_nomad_job_status_dead gauge
# HELP nomad_nomad_job_status_pending nomad_nomad_job_status_pending
# TYPE nomad_nomad_job_status_pending gauge
# HELP nomad_nomad_job_status_running nomad_nomad_job_status_running
# TYPE nomad_nomad_job_status_running gauge
# HELP nomad_nomad_job_summary_complete nomad_nomad_job_summary_complete
# TYPE nomad_nomad_job_summary_complete gauge
# HELP nomad_nomad_job_summary_failed nomad_nomad_job_summary_failed
# TYPE nomad_nomad_job_summary_failed gauge
# HELP nomad_nomad_job_summary_lost nomad_nomad_job_summary_lost
# TYPE nomad_nomad_job_summary_lost gauge
# HELP nomad_nomad_job_summary_queued nomad_nomad_job_summary_queued
# TYPE nomad_nomad_job_summary_queued gauge
# HELP nomad_nomad_job_summary_running nomad_nomad_job_summary_running
# TYPE nomad_nomad_job_summary_running gauge
# HELP nomad_nomad_job_summary_starting nomad_nomad_job_summary_starting
# TYPE nomad_nomad_job_summary_starting gauge
# HELP nomad_nomad_job_summary_unknown nomad_nomad_job_summary_unknown
# TYPE nomad_nomad_job_summary_unknown gauge

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions