supervisor: mark deliberate severance so a probe-induced death is not a crash (#64)

Merged
IC
3mo ago
+523-24

Mechanism only, ahead of the Tier-2 probe, per the ruling on #64.

A probe that induces a fatal connection state kills the module, and today that death is indistinguishable from a genuine crash at on_child_exit — ExitReport carries no reason. So the probe would consume restart budget and could push a module to Failed while measuring whether it decodes a flag bit.

The three requirements

1 — severance bound to (pid, start_time), not module id. record_deliberate_severance stores a ProcessIdentity and the child-wait path compares the exited child's identity before classifying. The race arm is the point of it:

let severed   = ProcessIdentity { pid: 41, start_time: 101 };
let successor = ProcessIdentity { pid: 41, start_time: 202 };
assert!(!module.record_deliberate_severance(severed).unwrap());
assert_eq!(exit_report.kind, ExitKind::Crash);

Same pid, different start time — the successor's death is a crash, not a laundered exemption. Same identifier-reuse discriminator as #76, reusing that PR's process_start_time rather than adding a second reader.

2 — budget/lifetime divergence, asserted in both directions. A deliberate severance increments lifetime_restarts and leaves restart_count alone. Both counters are asserted in the same test, and the crash arm asserts both move — a change that exempts everything passes the first arm while destroying the budget, so the second arm is what keeps the first honest.

3 — decode tolerance. TerminalExitKind is an open string enum with an Unknown(String) variant: an unrecognised wire value decodes to Unknown("future_exit_kind") rather than failing the record.

A note on that third requirement, because it bears on #79

Unknown(String) retains the unrecognised value rather than collapsing it. That is what makes it different from the catch-all docs/subc-control-protocol.md:275 rejects — the objection there is that a fallback makes an unknown op and a malformed body decode alike, and those warrant opposite responses. An open variant carrying the string preserves that distinction: an unknown kind arrives as Unknown("..."), while a malformed body still fails to decode.

I raise it because #79 asks the same question about RunningImageUnavailableReason, which shipped closed in #76 and breaks 0.6.0 clients on the whole supervisor.provenance response. This PR demonstrates the middle path exists and costs one variant. Whichever way you rule on #79, the two enums should end up on the same principle rather than differing by which PR they arrived in.

Platform limitation — states plainly, because the probe author needs it

process_start_time reads /proc/<pid>/stat and returns None off Linux. No identity means no severance marker, so on macOS and Windows a probe-induced death is classified Crash and consumes restart budget.

That is the correct failure direction — fail closed, never grant an exemption we cannot substantiate — and it matches the house rule that an unconfirmable identity forfeits the claim. But it is a real constraint on the probe: on the macOS fleet, Tier-2 adoption checking spends budget. Worth knowing before the probe is designed rather than discovering it when a module hits Failed.

A macOS path would need a different start-time source (sysctl KERN_PROC_PID / kinfo_proc.ki_start); out of scope here.

Naming

DeliberateSeverance names the mechanism — the daemon attests that it severed the connection, not why. The doc comment says so, so a future non-probe deliberate severance either reuses it honestly or adds its own variant instead of borrowing a probe-shaped label.

Verification

cargo test --workspace                    779 passed  0 failed  1 ignored
cargo clippy --workspace --all-targets -D warnings   clean
cargo fmt --all --check                   clean
cargo build --workspace --locked          clean
check-wire-crate-versions.sh origin/master  6 crates examined, none unbumped

Baseline on 6cf83890 is 775, not the 774 I briefed — 775 + 4 new tests = 779. The implementer measured 779, flagged that it could not reproduce my number, and reported the measurement instead of adopting the brief. My error, caught by refusing to launder it.

Red proofs, one per arm: deliberate branch spending budget → expected 0 got 1 · crash branch omitting the increment → expected 1 got 0 · identity match removed → stale-marker assertion failed · deliberate kind serialised as crash → expected deliberate_severance got crash.

Versions: subc-control 0.8.0, subc-client-rs 0.10.0, subc-core 0.10.0. One golden fixture updated — client_control_response_supervisor_terminals.json now pins exit_kind: crash on real-handler terminal records, which is the pre-existing behaviour made explicit rather than a change to it.


View with [code]smith Autofix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need. Autofix is disabled.


Summary by cubic

Fixes probe-induced module deaths being classified as crashes: a probe that kills a module via fatal connection severance now gets a deliberate severance marker, so the death no longer consumes restart budget.

  • Deliberate severance is bound to (pid, start_time) so a successor process with a reused PID is still treated as a crash.
  • A deliberate severance increments lifetime_restarts but leaves restart_count unchanged.
  • The new exit_kind wire field is additive and decodes unknown future kinds as Unknown(String) instead of failing the record.
  • Generic fatal connection teardown cannot arm the marker, so a surviving process can't carry a stale exemption into a later genuine crash.
  • On macOS and Windows, process_start_time is unavailable, so severance cannot be verified and probe-induced deaths still count as crashes and spend restart budget — fail closed.

Consumer impact

  • ck module terminals now prints the exit kind.
  • Package versions bumped: subc-control 0.8.0, subc-client-rs 0.10.0, subc-core 0.10.0.