What you are looking at

Every node has two wells: CPU and memory. A pod's requests — not what it actually uses — decide how much of each well it takes, which is exactly how the real scheduler reasons. For each pending pod, kube-scheduler runsfilter plugins that throw out nodes that can't take it (not ready, wrong labels, an untolerated taint, not enough room), then score plugins that rank the survivors, then binds the pod to the winner. The badges above the nodes show each step; the event log underneath uses the same wording askubectl describe pod.

Taints, tolerations and affinity

gpu-1 carries a nvidia.com/gpu:NoSchedule taint, so ordinary pods are repelled from it. The ml pod tolerates the taint and requires accelerator=gpu through node affinity — a toleration only allows a pod onto a node, it never pulls it there. postgres requires disk=ssd, so it can only land on node-b, and if that node is full it stays Pending no matter how empty the others are.

Failures and the autoscaler

When a node stops heartbeating it turns NotReady, and after a grace period its pods are evicted; their controllers create replacements, which go back through the queue. If nothing fits, pods stay Pending with a FailedScheduling event — the cluster autoscaler watches for exactly that, simulates whether a new node from a node group would help, and only then adds one. It won't help a pod whose constraints no node group can satisfy, and it removes nodes that sit empty for long enough.

Spread scoring (LeastAllocated) keeps headroom on every node; bin-packing (MostAllocated) fills nodes up first so the autoscaler can remove the empty ones — cheaper, but a single failure then displaces more pods. The timings here are compressed from minutes to seconds.