fix: stop leaking informers and their caches after tracking ends
Backport of #406 to 0.
Summary
A long-lived process that tracks resources over and over (for example a controller that runs a Helm-compatible deploy per reconcile) keeps shared informers alive after tracking ends, each holding the cache of a resource type for a whole namespace, until the process restarts. A goroutine dump shows informers that got the stop signal but wait in wg.Wait forever, and client-go listeners stuck in chansend inside ReplicaSetInformer.Run. On 0 it happens on almost every Deployment tracking: the ReplicaSet informer replays an Add for every ReplicaSet of the Deployment into a channel with capacity 1 while the tracker is busy reporting readiness, and stops reading right after.
What
- After a tracker stops (its context is canceled or its informer cleanup is called), informer event handlers stop waiting on tracker channels, so the shared informer stops on the factory stop channel and releases its cache. Applies to the Pod, Deployment, StatefulSet, DaemonSet, Job, Canary and ReplicaSet trackers, the generic resource state and events watchers, and the event informer.
- While the tracker keeps reading, every event is delivered as before: channel capacities (including the ReplicaSet channels, capacity 1) and send order are unchanged, nothing is dropped.
- A fatal watch error is no longer sent to the factory's watch error channel once the factory is stopped. Before, a full channel nobody read blocked the reflector and kept the informer alive the same way.
- When starting a Deployment, Job, DaemonSet, StatefulSet or Pod tracker fails halfway, the informers it already started are released.
- New exported helper
utils.SendOrDone(ctx, ch, v): sendsvor gives up oncectxis done. - Not changed: the ReplicaSet failure handling from 38643e9 (fail fast on pod-create errors) is not backported; only the cancellation-aware sends are.
- Not changed: sends from tracker loops to consumer-facing channels (
Ready,Failed,Status,PodError,AddedReplicaSet, …) still block when the consumer is gone. They can leave a tracker goroutine behind, but not an informer and its cache. - UNVERIFIED: not reproduced against a real cluster. Every handler is covered by a regression test on a fake dynamic client that shows the informer goroutines stay alive without the fix and exit with it.
Why
Informer handlers run in the client-go listener goroutine. A handler blocked on a channel send keeps sharedProcessor.run in wg.Wait() after the stop, so sharedIndexInformer.Run never returns and the indexer stays referenced. On 0 no handler, including the ReplicaSet one, was cancellation-aware. Porting 38643e9 instead was rejected: it also changes Deployment failure semantics, which a maintenance branch should not pick up for a leak fix.