Recording rule for expensive ratio prometheus

precompute SLO error ratio to keep dashboards and alerts fast

groups:
- name: slo
  rules:
  - record: job:request_errors:ratio5m
    expr: sum by(job)(rate(http_requests_total{code=~"5.."}[5m])) / sum by(job)(rate(http_requests_total[5m]))

#prometheus #gitops #observability #performance

more prometheus all 30 commands →

Validate rules and config with promtool promtool check rules rules.yml && promtool check config prometheus.yml Instant query from CLI promtool query instant http://localhost:9090 'topk(5, sum by(pod)(rate(container_cpu_usage_seconds_total[5m])))' Find cardinality offenders in TSDB promtool tsdb analyze /prometheus --limit 20 Query Prometheus via curl curl -sG 'http://prometheus:9090/api/v1/query' --data-urlencode 'query=up' | jq . Instant query via promtool promtool query instant http://prometheus:9090 'up{job="kubelet"}'