fix(cmux-tui): a host reconnect ends the output store's tail, so a kill right after it restores the gap's output
Summary
Follow-up of #31643 (REOPEN-F3), from its review: a host reconnect inside a running daemon (the host closed the daemon's tap after its 8 MiB queued-output budget, for example while the daemon was stopped) replaces the daemon's terminal from the host's snapshot. The output store was not told. Bytes the host wrote while the daemon had no tap never reached the tail, and the tail kept continuing the old snapshot. A kill of every process before the next coalesced journal checkpoint restored the screen without that output.
Fix: OutputStore::stream_gap, called right after the reconnect records its host_reconnect journal gap. In snapshot mode the tail's unwritten bytes fall back to the snapshot (the one rule of #31643's failure-mode table), and the next writer pass writes a snapshot of the replaced terminal (after a write backoff when one runs). Full mode is unchanged (its segment reads report the gap by offset).
Worst-case loss after a reconnect: now the same as #31643's normal row (the last 100 ms plus one writer pass), not up to the checkpoint interval.
Tests
Command: cmux-ci run --class isolated --script scripts/ci/cmux-tui-rust-check.sh --ref SHA --os macos --arg test --arg <filter>.
- New:
kill_all_right_after_a_host_reconnect_restores_the_output_of_the_gap(terminal_host_recovery/kill_all_restore.rs). It stops the daemon while the shell writes 12 MB (the host closes the tap), resumes it, waits for the reconnect, then kills everything 0.8 s later and checks that the restored screen has the line printed during the gap. - Red at c2a18031d5c (feat + the test): step 996d6fe1b2ddddda25d5da4b FAIL ("the output written while the daemon had no tap is not restored"). Also red at 8d488fff913 on #31643's head: 7add29c543a4e840e9cf43b5.
- Green at f7cc9ea8102:
kill_all_restore6 of 6 (2bf6acccb780abe3edb87301),output_store41 of 41 (cd30318673d03491baeca8ce),reconnect_wave2 of 2 (3d9b478be7142f9cca79e0e9).
Observed / Not run
- Observed: the fleet runs above (macOS 26).
- Not run: clippy. The fleet clippy step is red on the base for a file this PR does not touch (
mux/agent_chat_notices.rs, from #31770; step b116a7adc4127c13ad43cf52). This PR adds 31 lines inoutput_store.rsandsurface/hosted_reader/reconnect.rs. - Not run: the app. No app code changes.
Risk
risk:normal: daemon logic, no file deletes, no protocol or schema change.
Changelog
Fixed: after a crash right after a terminal's background host reconnected, the terminal's last output is restored too.
![View with [code]smith](https://pr-comments-assets.blacksmith.sh/codesmith/view-with-codesmith-dark-v3-3x.png)
![Autofix with [code]smith](https://pr-comments-assets.blacksmith.sh/codesmith/autofix-with-codesmith-dark-3x.png)
Need help on this PR? Tag @codesmith-bot with what you need.
Summary by cubic
Fixes a bug where a host reconnect inside a running daemon left the output store's tail continuing the old snapshot, so a kill shortly after the reconnect restored the screen without the output written during the gap.
- Calls
OutputStore::stream_gapwhen the reconnect records its journal gap; in snapshot mode the next pass snapshots the replaced terminal, and full mode is unchanged. - Counts stream gaps in
LogStateso a snapshot captured before the reconnect but recorded after it still asks for the replaced screen. - Adds a test that stops the daemon while the shell writes 12 MB, resumes it, waits for the reconnect, kills everything 0.8 s later, and verifies the gap's output is restored.
- Adds a test that a reconnect during a snapshot write still asks for a snapshot of the replaced terminal.