Check conntrack stats on a node

Check conntrack drop and insert_failed counters per CPU; a full nf_conntrack table drops packets

kubectl debug node/<node> -it --image=nicolaka/netshoot -- conntrack -S

tool
k8s-debug27 commands
tags
k8s-debugnetworktroubleshootingkernel

More k8s-debug commands

  • kubectl run dnstest --rm -it --image=nicolaka/netshoot --restart=Never -- dig +short <name>.<ns>.svc.cluster.local

    Resolve a Service FQDN from a temporary pod to tell CoreDNS problems from app DNS problems

  • kubectl events --for pod/<pod> -n <ns> --types=Warning

    Show only Warning events for one pod, in time order; the newer way instead of get events with filters

  • kubectl logs <pod> -n <ns> -c <name> --previous --timestamps --tail=200

    Show the last 200 lines of the previous container instance, with timestamps, to catch the stack trace before the restart

  • kubectl debug -it <pod> --image=nicolaka/netshoot -n <ns>

    Attach a netshoot debug container to a running pod; it shares the pod network namespace

  • kubectl top nodes

    Show CPU and memory for each node (needs metrics-server)

all 27 k8s-debug commands