acpmux peer snapshots (3): what starts or grows a copy

Merged
TE
9d ago
+719-16

Lawrence's list item, word for word: "start new VMs from acp chat - decouple persistence of acp trajectories from the VM itself, in case the VM is resuming".

Part of the client-snapshot piece of manaflow-ai/cmuxterm-hq#4381 ("Persistence, pause and resume, and history hydration"): the Mac's acpmux keeps a durable, bounded copy of each peer session's events, so the app paints a paused or offline peer chat at once, marked possibly stale, and catches up from the host with afterSeq. The host's journal stays the authority; the copy is not a write lease, an execution checkpoint or a backup. The shared fenced-writer database from #4381 is not part of this work.

This replaces #25323 (store) and #25378 (hook and pane), split so each piece lands on its own under about 800 changed lines, tests included. The series:

  1. the on-disk form of a copy (#25448, landed)
  2. the store's thread: bounds, writes, removals (#25445)
    2t. the store's tests (#25505)
  3. what starts or grows a copy: generations, joins, links (#25446)
  4. purges, removed peers, the queue's bound (#25447)
  5. the hub's taps (wiring) (#25449)
  6. the read hook _acpmux/peer_snapshot, which the VM start/resume work calls too (#25473)
  7. the pane's offline banner and the reply's reading (#25474)
  8. the pane paints the copy, then catches up (#25475)
    8t. the pane's tests (#25506)
    8t2. tests of 8's attach fixes (#25620)

This PR (3): what starts or grows a copy

What starts or grows a copy:

  • Generation. Only a session's listed generation starts or grows a copy. That generation is the host's createdAt for the session, so a recreated session never merges into an old copy.
  • Link. Every page and live record carries the link it came on: the peer's instance and the link's epoch. Anything from an older link than the newest one seen is dropped.
  • Pages.
    • A copy covers first_seq..=last_seq.
    • A page joins a copy only where the two meet. Otherwise it is ignored, or it replaces the copy.
    • A page from a peer that went back below what the copy saw replaces the copy, even when the page is empty.
    • A page with hasMore: true covers only up to its last record.
  • Live records. A live record joins only a copy that a page joined on the same link, and only at one past last_seq. After a lag of that link, no live record of it is taken until a page comes; a lag notice of an older link, taken late, leaves a copy a newer link's page joined (Op::Lagged { peer, link }, S41).
  • How far a page covers. A page is one read of the peer's log, so every record of the kinds asked up to its last one is in it. Past that it covers only as far as the peer says it holds (through, from the pageThrough field that 5 adds to the attach and events replies). A peer that does not say (an older daemon reads lastSeq after its page) has its page cover its own records only, so a live record that came between can never be claimed without being held. A forward page cut by hasMore covers only its records. This replaces the Missed bookkeeping of #25323 (S15, S17, S26 to S29).
  • Watermarks. A page whose afterSeq, through or lastSeq is not below 2^53 is not taken, so no bound's + 1 can overflow (record seqs pass seq_of, below 2^53, S48). An empty page behind the copy is judged by lastSeq, else through. The copy keeps how far the peer was seen, its lastSeq and its through, from a page cut by hasMore too (S47).
  • Generations only grow. A row naming an older generation than the one listed (a list read before the change that named the newer, which a peer's watch reply can be) takes nothing back, so a page asked on it is never kept (W7).
  • Records from another session are never kept.
  • Names. A key whose id is too long for a file name is not taken.
  • Before the disk is read. What was unlisted before the disk read is applied to what the scan finds.
  • Memory. A copy a page starts is counted with its own cost (Store::insert), and again when its row changes (charge), as 2 requires.

Findings from the earlier reviews this piece answers

These are S12 to S15, S17, S19, D13 and G1 to G5 of the ledger (Codex gpt-6-astra xhigh, rounds 1 to 11 of #25323):

  • disjoint history joined;
  • live capture before continuity;
  • an events page with no cursor that claimed coverage to the end;
  • an attach lastSeq past its page;
  • a missed record below a page's newest seq taken as covered (round 11);
  • an id too long that made a copy that never writes (round 11);
  • an in-flight capture undoing a purge;
  • rollback detection, including an empty page;
  • link epochs sampled late;
  • generations unknown or not compared;
  • this PR's third review: a lag not scoped to its link (S41); the eviction and backoff findings (S42, S43) are fixed in 2;
  • this PR's fourth review and 5's fifth: a cut page's watermark (S47), record seqs (S48, by seq_of), an older list taken late (W7);
  • this PR's fifth review: a page with a record seq out of range is rejected whole (S52); an empty attach page that names its generation starts an empty copy (S53); a row over HEAD_BYTES removes its copy at once (S54);
  • this PR's sixth review: a lag advances the peer's newest link (S56); listed is bounded by LISTED_KEYS and keeps no entry for a session listed with none by a fully listed peer (S58); the scan's eviction before a first listing is answered (S57: it evicts only past the bounds, which a run of the same store never leaves);
  • this PR's seventh review: a listed id no file can have takes no entry (S63); the costs of ids are charged in 4, where the queue's bound lands (S64).
  • this PR's eighth review: a row after a full list that left its session out is the peer's newer word, taken in order (S71, answered).

Tests

Release compile: green on the build fleet at this head (8be738c84d8), fleet step 2cca981895951a50f635c927 (scripts/cmux-next/check-release-compile.sh, Xcode 26.6), in place of the Blacksmith job that stayed queued.

These 9 new tests (22 in all) pass on the fleet, with fmt and clippy -D warnings clean:

  • only the listed generation;
  • joins;
  • links and lag;
  • a peer that went back, including an empty page;
  • a page covers past its last record only as far as the peer says it holds;
  • another session's record;
  • unlisted before the scan;
  • a name too long.

Timers, threads and loops

  • The store's thread (acpmux-peer-snapshots, Store::run), added in 2. It blocks on its channel until an op comes or the next due write or retry is reached (recv_timeout), and it ends when every sender has dropped, that is, when the hub drops. It holds no timer of its own beyond that one wait.
  • Writes go out FLUSH_AFTER (2 s) after a copy changes.
  • Retries (a write that failed, a refused removal, a root that did not read) start at 1 s and double to 60 s. Each retry stops once it succeeds, or once its copy or path is gone. A flush never cuts a retry's backoff short; only the shutdown flush tries once more.
  • flush_now (shutdown, 5) waits for its answer at most ANSWER_WITHIN (5 s), and does so through block_in_place on the multi-thread runtime.
  • No other timers, observers or polling.

Base reds, not from this PR (they fail the same way on feat-cmux-next): agent_tools::tests::own_servers_run_under_the_daemon_prefix, and the flaky session_pool::pooled_sessions_end_with_the_daemon_and_after_a_crash.

Agent session: agent session

šŸ¤– Generated with Claude Code