Find OOMKilled containers
List pods in all namespaces where a container last exited with OOMKilled
kubectl get po -A -o jsonpath='{range .items[*]}{.metadata.namespace}{" "}{.metadata.name}{" "}{.status.containerStatuses[*].lastState.terminated.reason}{"\n"}{end}' | grep OOMKilled
- tool
- k8s-debug27 commands
More k8s-debug commands
-
kubectl get po -A --field-selector=status.phase=Pending -o name | xargs -I{} kubectl describe {} -n <ns> | grep -A3 FailedSchedulingShow the FailedScheduling reasons for every Pending pod (taints, resources, affinity)
-
kubectl describe node <node> | grep -A12 Conditions; ssh <node> 'journalctl -u kubelet -n 100 --no-pager'Read the node Conditions, then the last kubelet log lines, to find why the node is NotReady
-
kubectl get events -n <ns> --field-selector reason=Failed -o wide | grep -i 'pull\|manifest\|unauthorized'Show image pull errors from the registry: wrong tag, missing imagePullSecret or rate limit
-
kubectl get po -A | grep Evicted | awk '{print $2" -n "$1}' | xargs -L1 kubectl delete poDelete all Evicted pods left after node-pressure eviction
-
kubectl patch po <pod> -n <ns> -p '{"metadata":{"finalizers":null}}' --type=merge; kubectl delete po <pod> -n <ns> --grace-period=0 --forceClear finalizers and force-delete a pod stuck in Terminating; first make sure the node is really gone