acpmux: a peer chat opens at once on its copy while its peer reconnects (read peer chats ahead)

TE
7d ago
+899-9

Summary

Leo, 2026-10-30, opening a chat on a remote machine while the link to it was reconnecting: "AND IT DID IN FACT LOAD BUT TOOK 30+ F***ING SECONDS". The pane showed peer_unreachable: <machine> (reconnecting, last seen 10s ago: …) until the link came back.

The Mac's acpmux already keeps a durable copy of each peer chat (hub/peer_snapshot.rs, #25323..#25506), and the pane draws it at once under the " is offline. This is the copy this Mac kept… Last updated {time}" banner while the machine does not answer (peerCopy.ts, #25474). But the copy holds only what a client on this Mac read. A chat nobody opened here, such as a lane another agent started on that machine, has no copy, so it opens on the error until the link returns. A chat read hours ago has a copy that is hours old.

This PR changes the daemon so it reads peer chats ahead:

  • A daemon whose clients open peer chats (an attach forwarded this run) reads each peer's 24 most recently changed open chats (changed in the last 24 h). Closed, stale and empty chats are skipped.
  • When: a round starts 2 s after a peer's first row change, list or reconnect. One round runs per peer at a time, with one read at a time inside it. A round stops at the first read the link refuses; a chat the peer refuses on its own is skipped and read again after the interval.
  • How much: one page per chat per round, at most the pane's open page (400 records, 128 KB, the attach's kinds). Once a copy exists, a read asks only for what comes after it (afterSeq). It asks for the newest page when there is no copy, the copy is of another session, the copy is more than a page behind, or the copy is past the row (a peer back on an older disk, which the newest page then replaces). New records that don't fit one page are read on after the interval.
  • How often: each chat is read (or tried) at most once every 30 s per session generation, and its last change is always read when that interval ends.
  • Shutdown: the daemon's stop cancels every round before its last flush to disk, and no page is kept after the stop.
  • The page is kept the way any forwarded page is kept (capture_peer_page), with the same generation, link and ask-stripping rules. A newest page (beforeSeq past every seq, and no afterSeq) now counts as the chat's own run, like an attach's page. A page asked with both cursors is not kept, since it could skip records. As a side effect, the pane's hover read-ahead (#28239) also fills the durable copy.

Cost and staleness

bound
Diskunchanged: the copies' own bounds (2 MB + 256 KB per copy, 64 MB in total, least recently used first)
Memoryunchanged: 16 MB of records loaded plus 4 MB of heads; the read-ahead state is one Instant per planned chat (at most 24 per peer)
Network, first roundat most 24 × 128 KB per peer, once
Network, after thatonly records past each copy, at most 128 KB per chat per 30 s, and only for chats whose row changed
Staleness while the link is upa copy trails its chat by at most about 32 s (30 s interval + 2 s settle); a chat a client has attached is continued live, as before

Why the daemon and not the webview store: the webview's tail cache (tailCache.ts) lives only in memory and only for chats one page read, so it is gone after an app restart and empty for chats never opened. The daemon copy is durable, already bounded, already drawn by the pane, and already merged with the live page without a flash when the link returns (agent-pane-peer-copy.test.ts: "the machine coming back continues the copy where its page joins it, each record once").

What Leo's Mac showed

Coordinator's diagnostics from Leo's Mac, read-only:

  • ~/.acpmux/peer-snapshots is 63 MB, at the 64 MB bound.
  • The copy of the chat Leo opened is 2.1 MB with seqs 156247–157096. It was captured at 22:13Z, which is when Leo's own late open read the chat.
  • The live chat is at seq 166356, so that copy is now 9,260 records behind.

With this PR, the chat would have had a copy before the open, and it stays within about 30 s of live while the link is up.

Which part ships where

All changes are in the daemon (cmux-tui/crates/acpmux). No app or webview code changes: the pane's copy path and banner already exist. Leo's Mac runs daemon build c230bb58a, so this reaches it with the next NEXT nightly's daemon update.

Not in this PR:

  • The handshake failure (ao-rpc-size-audit, ao-peer-hb).
  • The error card's wording (queued-messages-strip).
  • Pane routing (ux-tabs-panes).

Performance

metricbeforeafterdeltaharness or run link
open-to-content, peer chat never opened on this daemon, peer downno content until the link returns (Leo: 30+ s)3 ms (5 records, from the copy)30+ s → 3 mscargo test -p acpmux --test peer_warm_copy -- --nocapture (two hubs, relay cut), local Linux run

Testing

  • cargo test -p acpmux --lib warm: 9 passed (adds the cut-short continuation, the stop, link-vs-chat refusals, the ahead copy and the per-generation throttle). Covers what a round plans (count, freshness, closed/stale/empty skipped, afterSeq vs newest, another generation), the 30 s interval, settle and one round per peer, a newest page being kept while an older beforeSeq page is not, and no read-ahead on a daemon whose clients open no peer chat.
  • cargo test -p acpmux --lib peer_snapshot: 62 passed. cargo clippy -p acpmux --lib --tests -- -D warnings: clean.
  • cargo test -p acpmux --test peer_warm_copy -- --nocapture: passed, open-to-content 1–3 ms across runs. Setup: B runs two chats and A mirrors B through a relay. A's client opens one chat, the other is read ahead, then the relay is cut. Opening the other chat then refuses the attach as peer_unreachable, and the copy holds its last reply.

Review

Astra (gpt-6-astra, high, fast), five rounds: BLOCK at a2cc1e2 (mixed cursors, cut-short catch-up, failures bypassing the throttle, rounds ignoring shutdown), BLOCK at 599fcb1 (stop/capture race, first-page failure never retried, page bursts), BLOCK at 112a7bf (a copy past its row, throttle not per generation), then SHIP at 5559a91. CI's panic ratchet then found this PR's own clippy::indexing_slicing in warm.rs (params["afterSeq"] = ...); 099eca7 inserts through as_object_mut and drops warm.rs's raw arithmetic, and round 5 is SHIP at 099eca7b11e950600990d8ac28b3b1dad025b7e6 (no behaviour change). Its one P3 from round 4: no test drives the ahead-copy replacement through Store::behind end to end (the planner side is tested).

Observed / Not run

  • Observed, on the lane VM at 099eca7: the three runs above and clippy. The ratchet's clippy (rust_panic_ratchet.py --flags) finds no hit in warm.rs and none on another changed line.
  • Observed, through cmux-ci at 5559a91 (run 41154152305): lib 2457 passed, 0 failed; peer_integration 51/51; peer_degraded 1/1; peer_warm_copy 1/1; clippy all-targets clean.
  • Observed, base red: cargo test -p acpmux --test peer_warm_copy with the PR's src reverted fails with no copy of a chat never opened here (the copy reply holds no events).
  • No capture: this PR changes no UI code. The "after" view is the existing peer-copy banner (PeerCopyBanner.gallery.ts, #25474). Not measured on Leo's Mac: that needs this daemon build there.

Changelog

Fixed: A chat on another machine opens at once on the Mac's copy while that machine reconnects, even if it was never opened on this Mac.

Agent session: agent session

🤖 Generated with Claude Code