List active alerts

Show only firing alerts with name and severity

curl -s http://prometheus:9090/api/v1/alerts | jq '.data.alerts[] | select(.state=="firing") | {name: .labels.alertname, severity: .labels.severity}'
tool
prometheus30 commands
tags
prometheusmonitoring

More prometheus commands

  • List all metric names
    curl -s http://prometheus:9090/api/v1/label/__name__/values | jq '.data | length, .[0:20]'

    Count total metric names and preview first 20

  • Series cardinality per job
    count by (job) ({__name__=~".+"})

    PromQL: count active time series grouped by job label

  • Silence an alert
    amtool silence add alertname=HighMemory --duration=2h --comment='Investigating'

    Create an Alertmanager silence for 2 hours

  • Check TSDB status
    curl -s http://prometheus:9090/api/v1/status/tsdb | jq '{head_chunks: .data.headStats.numChunks, series: .data.headStats.numSeries}'

    View TSDB head stats: chunks and active series

  • 95th percentile latency from histogram
    histogram_quantile(0.95, sum by(le,job)(rate(http_request_duration_seconds_bucket[5m])))

    p95 latency per job; note: always aggregate _bucket by le before histogram_quantile

all 30 prometheus commands