Run-queue vs cores

vmstat r column (runnable) above nproc means CPU saturation, not just high load

vmstat 1 5 | awk 'NR>2{print "runq="$1" blocked="$2}'; nproc
tool
troubleshooting32 commands
tags
troubleshootingperformancecpudebug

More troubleshooting commands

  • Process thread count
    ps -o nlwp= -p <pid>; ps -eLf | wc -l

    Thread count per pid and system-wide; runaway threads exhaust kernel.pid_max

  • RSS leak over time
    while sleep 5; do ps -o rss= -p <pid> | awk '{print strftime("%T"),$1/1024"MB"}'; done

    Sample RSS every 5s to confirm a real leak (monotonic growth) vs steady churn

  • Fast memory rollup
    cat /proc/<pid>/smaps_rollup

    One-shot Rss/Pss/Swap totals without parsing full smaps; cheap on huge maps

  • Largest memory mappings
    pmap -x <pid> | sort -k3 -rn | head

    Top RSS mappings by region; spot a leaking heap, mmap or shared lib bloat

  • Kernel slab top consumers
    slabtop -o -s c | head -20

    Sort slab caches by size; dentry/inode bloat explains kernel memory pressure

all 32 troubleshooting commands