|
upstream contribution · merged ✓
the ghost pod that crashed the controller
github.com/ovn-kubernetes/ovn-kubernetes ·
PR #6866
|
|
🐣 first — what is EVPN live migration?
KubeVirt runs virtual machines as pods on Kubernetes.
Live migration moves a running VM from one node to another.
The EVPN controller manages MAC/IP neighbor entries
so the network follows the VM to its new home.
It tracks a source pod (old node) and target pod (new node).
|
|
|
🐛 one problem found
|
1. 🔴 nil dereference — SIGSEGV crash loop
After migration completes, Kubernetes garbage-collects the source pod.
The EVPN controller calls shouldDeleteNeighbors()
which dereferences SourcePod.Spec.NodeName
— but SourcePod is nil.
SIGSEGV → ovnkube-node crash-loops → network ClusterOperator stalls → upgrades blocked
|
|
2. same pattern in the other direction (defensive)
shouldEnsureNeighbors() dereferences
TargetPod without a nil check.
Not yet triggered in the wild, but same shape — fixed preemptively.
|
|
|
|
💭 what changed?
|
before fix
migration completes
source pod GC'd
↓
reconcile fires
SourcePod = nil
↓
SourcePod.Spec.NodeName
↓
SIGSEGV 💥
ovnkube-node crash-loops
|
|
after fix
migration completes
source pod GC'd
↓
reconcile fires
SourcePod = nil
↓
nil? → return false
↓
skip safely ✅
controller stays up
|
|
|
|
💡 the rule
if a struct field can outlive its source, nil-check before you dereference.
Lifecycle transitions are where objects disappear —
the code that ran fine during steady state will crash
the moment the world moves under it.
|
|
|
merged: Aug 27 2026 |
repo: ovn-kubernetes/ovn-kubernetes |
pr: #6866
small fix · big lesson · contributed by Parikshit Khedekar
|
|