acpmux: a turn ends when Claude Code ends it

Merged
TE
5d ago
+2283-106

Leo, 2026-10-29: "why the f*** are you still depicted as thinking for 11 hours? This feels like we're not reading the data correctly. Because you're, like, you clearly just ended some fing turns, and it still fing shows this."

Leo, 2026-10-29: "What are we gonna do with these 200 fing messages? You never end a fing turn. As soon as it comes in, it fing, like, occupies you for another fing turn." and "you can't you just deal with this? Like ACPmux can't be this stuff. How do swarms work?"

What was wrong

A coordinator session's daemon turn started about 16.7 h ago and never closed. Claude Code ended many of its own turns (end_turn) inside it, but each time a steered message was still unread. Claude Code runs an unread steer as a turn of its own. The adapter treated that turn as more of the same prompt and answered the prompt only at a result with nothing unread. With messages arriving all the time, that never happened. The GUI's "Thinking 11h 13m" was showing that one daemon turn accurately.

The fix

A daemon turn now ends when Claude Code ends its turn.

  • Adapter (claude_stdio/carry.rs). A clean result (not stopped, not failed) that leaves steers unread now answers the prompt right there, marked _meta.claude.carried: {seq, promptIds}, and the adapter stays in-turn. A steer read after that is answered steered with _meta.claude.carried: seq. The hub's prompt for the next turn (_meta.carried: true) writes nothing. It is answered at that turn's result, or at once if that result came first.
  • Hub (hub/carried_turn.rs). When an answer is marked carried, finish_turn ends the turn. In the same step (take_turn) it puts the next turn in place with a new turnId and startedAt, so no prompt ever finds the chat idle between the two. The new turn's turn_started names carried.fromTurnId and the carried promptIds. Its prompt is the first carried steer. A task holds the turn lock for that turn and loops while Claude Code keeps carrying. A steer read after a carry gets the new turn's turnId in its answer, so a waiting acpmux send answers with the reply of the turn it actually went into (cli/steer_reply.rs).
  • Output hold (carry.rs holding). After a carry, the adapter waits to read Claude Code's next lines until the next turn's prompt arrives. The wait is bounded (HOLD_MAX, 10 s). The in-process reader (agent.rs) and the agent host (agent_host/host.rs, which also stops pacing stdout in while it holds) both do this. The hub logs the old turn_result and the new turn_started before it sends that prompt. So what Claude says in the new turn is logged inside the new turn, and reply_of_turn and the transcript give it to the right turn.
  • Stop during the hand-off (cancel.rs). A Stop captures the turn and then waits for the writer. If that turn carried on in the meantime, the Stop now follows the carried successors (SteerNow.successors) to the turn running now and interrupts it. A turn that started on its own, such as a queued prompt, is still left alone.
  • Lost result (Carry::fail_owed). When a carried turn's result is lost (an answer over the size limit) before its prompt arrives, the prompt now fails instead of ending as a success.
  • Later review rounds.
    • A steer still unread in a carried turn is answered when Stop forces that turn's end: agent_prompt_hook accepts a settlement for a turn carried on from the steer's own turn (Hub::succeeds).
    • Stop's forced end and its grace timer follow the carried turns too.
    • A restart in the middle of a hand-off loses nothing:
      • The ended turn's turn_result names the turn it carried into (carriedTo), and a carried turn's turn_started is marked.
      • On restart, adoption runs that turn on: it sends the carried prompt to the surviving host instead of recording "never received".
      • This covers a restart before the carried turn's prompt went out, and one before its turn_started was logged.
    • A sub-session's run slot (maxRunning) stays taken until the last carried turn ends.
    • A carried turn that a connection error cuts off goes on by itself (auto_resume.rs), as a prompt's own turn does. Its first steer's family sender and trust gate travel with it.
    • A carried turn recovered after a restart gets the recovered turn's remote-floor checks.
    • The hand-off keeps a Web steer's raised control, both live and on recovery. It also keeps a Stop's no-continuation mark, and the ancestry of steers still waiting.
    • A steer never publishes a carried successor's forced settlement a second time.
    • A prompt that waited on the turn lock while the turn before it carried on now queues behind the carried turn instead of replacing it.
    • While the daemon stops, a carry still names its next turn, and the next daemon runs it.
    • A dead-host continuation of a carried turn quotes the steers it carried, with their images, and keeps the trust gate.
  • What follows real turns now: the turn timer, lastTurn / "Read after this turn", lane reports at turn end (one per real turn), the session status, and auto-resume marks. Queued prompts still wait while Claude Code runs a carried turn: Claude is busy, and writing a prompt then would collide with it.
  • Unchanged:
    • A steer Claude Code reads mid-turn still joins that turn.
    • Stopped and failed turns, and the step that Send now interrupts, still run an unread steer as one more turn of the same prompt (steer_now.rs).
    • codex-acp steers: no _meta.claude.carried, so steer_end.rs's answer fan-out is untouched.
    • Joined digest replies and requeue on failure: carried turns only start from an Ok answer.
    • Each steer is still answered exactly once, at its echo.
  • A turn recovered from an agent host after a daemon restart (hosts.rs) carries the same way.

Tests (red first)

  • claude_stdio/carry_tests.rs:

    • an unread steer at end_turn ends the prompt and starts the next turn;
    • a carried turn whose result arrived before its prompt answers at once;
    • three ends in a row while steers keep arriving give three carries;
    • a steer read mid-turn joins the turn;
    • a failed turn keeps the old path.

  • tests/hub_integration/carried_turns.rs, using fake_claude's new gated: PATH turns:

    • a steer read after end_turn starts a new turn with its own turnId and a later startedAt, and the steer's answer names that turn;
    • four Claude turns produce four turn_started/turn_result pairs, each steer answered once with its turn;
    • a steer read mid-turn stays in one turn;
    • a codex-acp turn that took two steers is still one turn.

  • hub/carried_turn.rs unit tests: parsing of the carry markers.

  • claude_stdio/carry_tests.rs also covers: a steer written during a carried turn names it; a carried prompt with nothing owed ends at once; a carried turn whose result was lost fails its prompt.

  • tests/hub_integration/carried_turns.rs: what Claude says in the new turn (read: LATE-1) is logged after the first turn's turn_result, between the new turn's turn_started and its turn_result.

  • tests/agent_host/carried_hold.rs: a Claude agent host reads nothing more after a carry until the next turn's prompt arrives, then logs the new turn's output.

  • hub/carried_turn.rs: a Stop that saw the turn before a carry (or two carries) stops the carried turn, but not a queued prompt's turn.

  • tests/hub_integration/carried_turns.rs: Stop on a carried turn whose Claude answers the interrupt but runs on. acpmux ends the turn itself, and the steer still unread in it is answered cancelled.

  • tests/hub_integration/carried_turns.rs: a carried turn that fails with "API Error: Connection error." schedules its auto-resume and runs p-late~resume-1.

  • tests/hub_integration/carried_turns.rs: Claude reads two steers during its turn, then sends one end_turn. That end_turn ends the daemon turn, both steers name that turn, and the 6 queued prompts each run as their own turn. The chat ends with no turn and is not stuck. A second variant does the same with a sub-session's next_step result as one of the steers. Both reproduce a coordinator chat's report (2026-10-29: "even if im just queueing, your turn should still be ending").

  • hub/hosts_tests.rs: adoption finds a carried turn whose prompt had not gone out, whether or not it logged its start.

  • tests/durable_agents/carried_restart.rs: the daemon is killed in the middle of a hand-off (debug seam ACPMUX_TEST_CARRY_HOLD_MS). The next daemon runs the carried turn to completion on the surviving host, and its words are logged inside it.

  • Red at 9532b62cbd1 (tests only, on base): https://github.com/manaflow-ai/cmux/actions/runs/40937666975. 4 of the 5 carry_tests fail. carried_turns fails a_steer_claude_reads_after_end_turn_starts_a_new_turn (the turn never ends) and turns_claude_ends_while_steers_keep_coming_are_separate_turns ("never 2 turns"). The two "unchanged" checks (mid-turn join, codex) pass on base, as they should.

  • Green at aed4992419f (the current head: feat-cmux-next merged in, conflicts in queue.rs and steer_end.rs resolved, and a carried turn arms its own turn-cap timer): https://github.com/manaflow-ai/cmux/actions/runs/40981192638. clippy is clean. agent_host 23/23, send_steer 5/5, lane_reports 17/17, subsession_runs 1/1, durable_agents 125 passed with 0 failed (base's 8 MiB stack fix cleared the old 9). --lib 1986 passed, hub_integration 564 passed. The carried_turns/steer/held_queue repeats pass 36/36 four times, and carried_restart passes. The only failures also fail on base b6554ad89da, with the same test, line and error (run https://github.com/manaflow-ai/cmux/actions/runs/40982455847):

    • daemon::state_file_tests::managed_unit_start_avoids_a_symlinked_default_home (state_file_tests.rs:87, Text file busy; flaky on base too, 1 of 3)
    • auto_resume::retry::stop_before_a_retrys_turn_starts_prevents_it (auto_resume_retry.rs:222)
    • transcript_events::orchestration_fields_reach_subagent_events_and_session_summary (transcript_events.rs:721)

  • Green at b9b5bc72569 (it adds only tests after 938e81f73ab; it adds only tests after 938e81f73ab): https://github.com/manaflow-ai/cmux/actions/runs/40973237771. clippy is clean, and the 9 carried_turns tests pass 3/3.

  • Green at 938e81f73ab: https://github.com/manaflow-ai/cmux/actions/runs/40970689866. Only the base reds fail. --lib 1921 passed, hub_integration 550 passed, carried_turns 4 repeats green.

  • Green at 6c8ebd897d8: https://github.com/manaflow-ai/cmux/actions/runs/40966627705. The same suites and repeats as below give the same results: only the base reds fail. --lib has 1919 passed, including the new succession and Stop-mark tests.

  • Green at aa603f9f20e: https://github.com/manaflow-ai/cmux/actions/runs/40960986752.

    • clippy -D warnings is clean.
    • agent_host 23/23, send_steer 5/5, lane_reports 17/17, subsession_runs 1/1.
    • Four repeats of carried_turns, steer and held_queue (33 tests): all green.
    • durable_agents::carried_restart: passes.
    • --lib 1917 passed, hub_integration 549 passed. durable_agents fails only its 9 base reds, at the same lines; the other two suites fail only the base reds listed below.

  • Green at 54e594339cb (before the last round's auto-resume and floor fixes): https://github.com/manaflow-ai/cmux/actions/runs/40957168744.

    • clippy -D warnings is clean.
    • hub_integration: 548 passed. Only the 2 base reds failed.
    • lib (hosts, carry, cancel, steer): 207 passed.
    • carried_turns: 5/5 repeats green.
    • durable_agents::carried_restart: passes.

  • The full suite at 3bc2ebba9 (run 40956163014) gave the same picture:

    • agent_host 23/23, send_steer 5/5, lane_reports 17/17, subsession_runs 1/1.
    • durable_agents: 116 passed. Its only failures were the 9 base reds, at the same lines.
    • The one failure there, the new forced-Stop test, was a race in the test itself: it read turn_result before the carried task had logged it. It is fixed at the current head.

  • Green at 08603fc70b9 (the current head): https://github.com/manaflow-ai/cmux/actions/runs/40951775144. clippy is clean; agent_host 23/23, send_steer 5/5, lane_reports 17/17; --lib, hub_integration and durable_agents fail only the base reds listed below. One run also failed prompt_supersede::a_prompt_queued_before_a_restart_is_still_replaced (the cut turn's resume, under full-suite load). It passes 20/20 at head (run 40952662468) and 20/20 on base (run 40952691569). In 3 more full hub_integration runs at head (run 40953163684) it passed every time, and each run had 547 passed with only the 2 base reds failing. Its chat runs the plain ACP fake agent, so the carry and the hold never come into play.

  • Green at 14a783fe097 (the commit before; the head adds only the in-process reader's once-per-hold wait): https://github.com/manaflow-ai/cmux/actions/runs/40950658819.

    • cargo clippy -p acpmux --all-targets -- -D warnings is clean.
    • agent_host 23/23 (including carried_hold), send_steer 5/5, lane_reports 17/17, forum 21/21.
    • Four repeat passes of carried_turns, held_queue and steer (31 tests), carried_hold, and the lib's carry, carried and cancel tests (49): all green every time.
    • --lib 1916 passed. Its 1 failure, protocol::tests::protocol_ir_is_current (stale committed IR), fails the same way on base (run 40940204206).
    • hub_integration 547 passed. Its 2 failures, auto_resume::retry::stop_before_a_retrys_turn_starts_prevents_it (auto_resume_retry.rs:222) and transcript_events::orchestration_fields_reach_subagent_events_and_session_summary (transcript_events.rs:721), fail at the same lines on base (run 40941648353).
    • durable_agents 115 passed. Its 9 failures are the same 9 tests at the same lines as base (durable_agents.rs:311 Ɨ8, :238 Ɨ1), with the same daemon stack-overflow aborts (base run 40946940496).

  • Earlier head 945eb11a629: https://github.com/manaflow-ai/cmux/actions/runs/40942431588.

After deploy

The adapter runs inside the agent host process. A host that survives a daemon update keeps the old adapter, so a turn that is already stuck on such a host does not close by itself. It closes once that chat's agent host runs this build. New turns on new hosts follow Claude Code's turn ends.

Risk

Medium. This changes how Claude turns end when steers are pending. It touches the shared turn-end path (finish_turn now takes the next turn) and the Claude adapter's result handling. The fallback for a missed hand-off is bounded: a steer whose new turn is never named answers with its old turn id after 10 s. P2 (coordinator inbox mode) and P3 (turn cap) build on these turn boundaries. The output hold is bounded too: if the next turn's prompt never arrives, Claude's output is read after 10 s anyway, and at EOF it is always read.

Size: about 1,800 changed lines. The adapter carry, the hub's carried turn, the hold and the Stop hand-off form one turn-boundary change: each half alone would leave turns misattributed or a Stop lost, so it is not split.

Agent session: agent session

šŸ¤– Generated with Claude Code