Per-core CPU saturation

Per-core %usr/%sys/%iowait — spot a single saturated core vs balanced load

mpstat -P ALL 1
tool
troubleshooting32 commands
tags
troubleshootingperformancecpumonitoring

More troubleshooting commands

  • Run-queue vs cores
    vmstat 1 5 | awk 'NR>2{print "runq="$1" blocked="$2}'; nproc

    vmstat r column (runnable) above nproc means CPU saturation, not just high load

  • Process thread count
    ps -o nlwp= -p <pid>; ps -eLf | wc -l

    Thread count per pid and system-wide; runaway threads exhaust kernel.pid_max

  • RSS leak over time
    while sleep 5; do ps -o rss= -p <pid> | awk '{print strftime("%T"),$1/1024"MB"}'; done

    Sample RSS every 5s to confirm a real leak (monotonic growth) vs steady churn

  • Fast memory rollup
    cat /proc/<pid>/smaps_rollup

    One-shot Rss/Pss/Swap totals without parsing full smaps; cheap on huge maps

  • Largest memory mappings
    pmap -x <pid> | sort -k3 -rn | head

    Top RSS mappings by region; spot a leaking heap, mmap or shared lib bloat

all 32 troubleshooting commands