acpmux: a peer chat whose attach fails is attached again, and says why

Merged
TE
6d ago
+188-10

The link's own attach of a peer's chat (hidden_attach) dropped its error in two places, and the chat stayed marked attached either way:

  • after a reconnect, the handshake task re-attached every marked chat with let _ = me.request(MUX_ATTACH, ā€¦), one after another, each with the 600 s request budget;
  • before a prompt, server/peer_forward.rs spawned the same attach with let _ =.

A chat whose attach the peer refused (it is gone, or the peer is on a build that refuses it) stayed in Peer::attached, so no later prompt ever attached it again (a mark is sent once): it never streamed on that link again, and nothing in the log said why. This is the same class as the handshake error #28105 fixes: an RPC error in a spawned link task that went nowhere while the other side waited.

Peer::attach_hidden is now the one way the link attaches a marked chat, and it logs every failure at warn with the peer's error. Only a refusal by the peer (refuses_attach: not found, method not found, invalid params) unmarks the chat, so the next prompt attaches it again. Any other failure (the link dropped, a timeout, a full send queue) keeps the mark, and the next link's handshake attaches it again. Each mark carries a ticket, taken under the mark lock, that a client relying on it renews (mark_attached on a chat already marked), and a refusal unmarks only the mark its attach started with: a prompt that found the chat marked while its attach was out keeps the mark it relies on. After a reconnect the re-attaches run 16 at a time, so one chat the peer is slow on holds up no other's stream, and a link with many chats does not fill its send queue.

Test

cargo test -p acpmux --lib attach_tests:

  • a_chat_the_peer_refuses_to_attach_after_a_reconnect_is_no_longer_marked: a link with three marked chats to a fake peer that refuses one attach (not found) and fails another (internal error). Once the handshake's health read arrives (after every attach), the refused chat is unmarked, and the failed one and the third stay marked.

  • a_chat_marked_again_while_its_refused_attach_was_out_stays_marked: the fake peer holds a refusal until the test marks the chat again (a prompt relying on the mark), then refuses; the chat stays marked.

  • Red on the test commit ce9c861d (run 41025989967): a refused attach stayed marked: {"kept", "gone"}

  • Green on 075630b6 (run 41040637083, cargo test -p acpmux: the lib suite and every integration target pass except hermetic_daemon_guard, which fails the same way on base: tests/web_image_attachments.rs lacks the hermetic helper, fixed by #28150; the job's exit 1 is forced by the command so the log prints). The head dd8950d7 only moves the ticket's allocation under the mark lock (a review finding); both attach tests pass there locally, and its full run is run 41043368158.

Observed / Not run

Observed: the red and green runs above, on hosted Linux through cmux-ci test, and both tests locally on a Linux lane box. Read, not run: the prompt path (server/peer_forward.rs) has no test of its own; it calls the same attach_hidden the tests cover.

Other swallowed errors this audit found (not in this PR)

  • peer_add / peer_reconnect / peer_lease with wait give up after 30 s (PEER_SETTLE_BUDGET) and answer OK with no reason while the handshake (90 s budget) is still running (#28170).
  • _acpmux/briefs asks each peer in turn (600 s budget each) and leaves out a peer that fails without saying so (#28172); _acpmux/models leaves out a failing peer silently (10 s budget).
  • peer_health.rs turns any error from a connected peer into {"unsupported": true}.

Changelog

  • Fixed: a peer machine's chat whose stream could not be reattached after a reconnect now streams again on the next prompt.

Agent session: agent session

šŸ¤– Generated with Claude Code