acpmux: a reconnecting peer link reads a delta of the watch list, not the whole list
Summary
Leo, 2026-10-31: "why is there a 15mb cap and why are we not following protocol/transport best practices with compression/better formats".
Every time a peer link reconnected, it read the daemon's whole session list again. That is about 3,700 rows at 2-4 KB each on a fleet daemon, even when the link was down for a few seconds and almost nothing changed. This PR sends a delta instead, the way a resync protocol does after a client has a snapshot (watch_delta.rs):
- After it reads a whole list, a link keeps each row it read, with a digest: the first 8 bytes of the SHA-256 of the row's compact JSON.
serde_jsonsorts keys, so both sides produce the same bytes. - When it links again to a daemon that lists the
watchDeltafeature and serves it as a peer, its_acpmux/watchcarriesknown: {sessionId: digest}. The reply hasdelta: true, the rows that are new or whose digest differs, the known ids that are gone (removed), and anunchangedcount. - The link rebuilds the whole list from the rows it kept plus the delta, and publishes it as a whole list. The hub then replaces the peer's rows exactly as before. It has to: the hub marks those rows stale while the link is down, so patching them in place would leave unchanged rows marked stale.
- A digest the daemon still lists means its row is the one the link kept, whatever
session_changednotices came in between. A row that changed and then changed back is still correct. - A daemon that never heard of
known(every build before this one) ignores it and sends the whole list. A link served as remote (Web) never asks for a delta. - The delta is asked for first, with or without tools as the kept list was read. That includes a daemon whose list needed pages of rows (#28588's remembered path), so a large list is not read whole on every reconnect. A delta too large for one message falls back to the existing paths and the whole list.
Cost: the link keeps each row's compact JSON (about one copy of the list's text per peer), plus a 16-hex digest per row. The daemon reads and hashes its own list once per delta request, at reconnect only.
Builds on #28685 (compression), which lands first. This branch merged it to share the link's byte counter, so until #28685 lands, its two commits show in this diff.
Performance
| metric | before | after | delta | harness or run link |
|---|---|---|---|---|
| reconnect read, 3,000 own rows, nothing changed (JSON text) | 4,294,742 B | 4,568 B | -99.9% | watch_delta_link_tests (real listener, peer token) |
reconnect request size (known) | 0 | ~40 B/row (~150 KB for 3,700 rows) | +150 KB once | computed from the param shape |
Testing
Ran on this lane VM: cargo test -p acpmux --lib -- watch_delta compressed_link peer::handshake_tests list_reply_tests session_list_pages gives 22 passed after the review fixes (24 with session_list_pages in an earlier run; 30 at the merged head with ws_shutdown and protocol_ir_is_current). That includes the existing handshake and paging tests (#28182, #28542, #28588) and the new ones:
watch_delta::tests: a reconnect reads only new and changed rows and the removed ids, then makes the whole list again; a row that changed and changed back comes from the kept row; digests ignore key order; a whole-list reply is not a delta.server::watch_delta_link_tests::a_link_that_read_row_pages_asks_a_delta_when_it_links_again: a peer whose list needed pages of rows gets one delta request on reconnect,withTools: falseas its list was read, and no pages. It fails without the fix that puts the delta first and passes with it.server::watch_delta_link_tests::a_reconnecting_peer_link_reads_a_delta_and_publishes_the_whole_list: a real listener with peer-token auth and 3,000 sessions of its own. The listener restarts (Close 1012) and the same link reconnects. It publishes the whole list again, every row exactly once, from 4,568 bytes against 4,294,742 on the first link.
ACPMUX_PROTOCOL_BLESS=1 cargo test -p acpmux --lib protocol_ir_is_current and bun scripts/acpmux-protocol-codegen.ts regenerated spec/acpmux-protocol.json and the webviews' generated types for the new known, delta, removed and unchanged fields.
Observed / Not run
Observed: the tests above, on this lane VM.
Not run: the full acpmux suite, clippy and web lint (there are no node_modules here). CI runs them on this PR. Also not run: a live reconnect between two fleet daemons; the journal's connected to ... line shows its text_bytes once both ends run this build.
Review: Codex gpt-6-astra (high). It answered BLOCK twice. First, the remembered row-pages path skipped the delta; fixed, with the regression test above. Second, withTools for the next delta was recorded before a paged read finished; it is now recorded only after the list is read whole. It then answered SHIP at b09585592376c8b25a405f6fec2fd1a4d13038d1. After the merges of current feat-cmux-next and #28685's updated branch, a delta review answered SHIP at 4a35f473dffc73b7c7ac792dd3987b5ae9faf3a6.
Changelog
Changed: A peer daemon that reconnects reads only the sessions that changed, not the whole list.
Proof
No capture: daemon transport only, nothing on screen changes.
Agent session: agent session
š¤ Generated with Claude Code