CPU throttling stats (cgroup v2)

Count and total usec a cgroup was CPU-throttled by cpu.max quota

grep -E 'nr_throttled|throttled_usec' /sys/fs/cgroup/system.slice/<name>.service/cpu.stat
tool
cgroups8 commands
tags
cgroupscpuperformancetroubleshooting

More cgroups commands

  • Read and set CPU quota (cgroup v2)
    cat /sys/fs/cgroup/system.slice/<name>.service/cpu.max # "max 100000" = unlimited; "50000 100000" = 0.5 core

    cpu.max is 'quota period' in usec; 'max' = no limit. note: prefer systemctl set-property

  • Per-device IO pressure and stats
    cat /sys/fs/cgroup/system.slice/<name>.service/io.pressure /sys/fs/cgroup/system.slice/<name>.service/io.stat

    PSI io stall plus per-major:minor rbytes/wbytes/rios; pin noisy disk neighbors

  • List PIDs in a service cgroup
    cat /sys/fs/cgroup/system.slice/<name>.service/cgroup.procs; systemctl status <name>.service | grep CGroup

    Every PID currently in the service slice; cross-check with systemctl status tree

  • Full cgroup tree with empty groups
    systemd-cgls --all --no-pager; systemd-cgtop -d 2 --depth=3

    cgls --all shows empty cgroups too; cgtop ranks slices by live CPU/mem/io

  • Legacy v1 ad-hoc cgroup
    cgcreate -g memory:/<name>; cgset -r memory.limit_in_bytes=512M <name>; cgexec -g memory:/<name> <command>

    libcgroup v1 throwaway limit. note: v2 unified hierarchy uses systemd-run --scope -p

all 8 cgroups commands