Check TSDB status

View TSDB head stats: chunks and active series

curl -s http://prometheus:9090/api/v1/status/tsdb | jq '{head_chunks: .data.headStats.numChunks, series: .data.headStats.numSeries}'
tool
prometheus30 commands
tags
prometheusmonitoring

More prometheus commands

  • 95th percentile latency from histogram
    histogram_quantile(0.95, sum by(le,job)(rate(http_request_duration_seconds_bucket[5m])))

    p95 latency per job; note: always aggregate _bucket by le before histogram_quantile

  • Predict disk full window
    predict_linear(node_filesystem_avail_bytes{mountpoint="/"}[6h], 4*3600) < 0

    alert when linear trend predicts disk full within 4h based on last 6h

  • Detect dead target with absent
    absent(up{job="node"} == 1)

    fires when no series match; catches a job that vanished from discovery entirely

  • Rewrite label with label_replace
    label_replace(node_uname_info, "host", "$1", "nodename", "(.+)")

    extract/derive a new label via regex capture; useful for joins and tidy dashboards

  • Max of per-second rate over window
    max_over_time(rate(node_network_receive_bytes_total[5m])[1h:1m])

    subquery: peak 5m rate sampled each minute over the last hour; spot bursts

all 30 prometheus commands