fix(membership): retain recently-declared-dead entries during pruning
Problem
We recently added MaxDefunctSiloEntries to cap the number of non-active (e.g. Dead) entries kept in the membership table. When the cap is exceeded, the cleanup agent prunes the oldest defunct entries, keeping the most-recently-updated ones so they still appear in membership snapshots.
Recency was measured by EffectiveIAmAliveTime (the later of StartTime and IAmAliveTime). The problem is that declaring another silo dead does not update that silo's IAmAliveTime — only a silo's own periodic self-update does. So a silo that was declared Dead just now, but last reported itself alive long ago, looks like the oldest defunct entry and is pruned immediately, disappearing from snapshots right after it died.
Solution
Reference the entry's suspect times during pruning instead of bumping IAmAliveTime on death. A new internal MembershipEntry.EffectiveUpdateTime returns the later of EffectiveIAmAliveTime and the most recent SuspectTimes vote. Declaring a silo dead records a suspect vote, so a recently-declared-dead entry is now correctly treated as the most-recently-updated defunct entry and is retained.
EffectiveUpdateTime is used consistently in:
- the cleanup agent's defunct-entry selection (
DefunctSiloEntryPriority) and cutoff computation, and - the in-memory membership providers' removal predicate (used by the default/dev
SystemTargetclustering).
This keeps the death-declaration timestamp as the source of truth without mutating IAmAliveTime.
Notes / scope
- Cloud/ADO.NET providers are unchanged. The Azure table provider already keys removal off the row write timestamp (which reflects the death write), so recently-dead entries are retained there. The ADO.NET provider keys removal off the
IAmAliveTimecolumn and cannot see suspect times without a schema change — that remains a pre-existing limitation and is out of scope here.
Tests
Added MembershipTableCleanupAgent_ThresholdCleanup_RetainsRecentlySuspectedEntries, which sets up two Dead entries where the one with the oldest IAmAliveTime has a very recent suspect vote, and asserts it is the entry that survives pruning. Full membership suite passes (67 tests).