Disk usage alert rule prometheus
PromQL expression: fire when root disk > 85% used
100 - (node_filesystem_avail_bytes{mountpoint="/"} / node_filesystem_size_bytes{mountpoint="/"} * 100) > 85
more prometheus
all 30 commands →
Reload Prometheus config
curl -X POST http://prometheus:9090/-/reload
List active alerts
curl -s http://prometheus:9090/api/v1/alerts | jq '.data.alerts[] | select(.state=="firing") | {name: .labels.alertname, severity: .labels.severity}'
List all metric names
curl -s http://prometheus:9090/api/v1/label/__name__/values | jq '.data | length, .[0:20]'
Series cardinality per job
count by (job) ({__name__=~".+"})
Silence an alert
amtool silence add alertname=HighMemory --duration=2h --comment='Investigating'