cmux NEXT daemon update: Cloud machines as update targets

Merged
TE
10d ago
+941-11

Part 9 of the rebuild of #23199 and #24070 (adopted by hb-acpmux). On feat-cmux-next. #25393 and #25394 below it landed (c743a343, 30f84dd5), and this branch is rebased onto them: head c9f1a3e3, the same patch-ids as the reviewed f3ea03e2.

Cloud machines as daemon update targets. Nothing offers them yet: the handshake follower and the header line are the next PR.

Leo: "BRO CANT YOU JUST LIKE SUSPEND SHIT AFTER A TOOL CALL/TURN THEN JUST GO AND AUTOUPDATE OUR TEAMMATES NEED ACPMUX UPDATING TOO..."

Leo: "OUR SHIT IS HARDLY EVER GOING TO GO IDLE. IT JUST WONT. YOU'LL JUST HAVE TO ACCEPT IT BREAKING AT SOME POINT BUT IT RESUMES WHATEVER IT INTERRUPTED. END OF STORY FOR THE DAEMON UPDATE."

Leo-ask: ask-48bbaec1d4
Leo-ask: ask-061e3574a6

Change

From #24070 at bd4892c8, on base's cmux link machine update --to-bundled (link/machine_update.rs):

  • CloudMachineUpdateTarget: the same cmux link machine update as Update Machine, with --to-bundled, so the machine gets the app's own pinned build. The CLI keeps the old binary, restarts only the daemon (terminals and chats keep running) and rolls back when the new one doesn't serve. A machine whose daemon the app refused goes first (urgent): nothing on it is in use through the app.
  • Never a move back. A Cloud daemon's build has no date, so the policy can't tell an older build from a newer one. The automatic update therefore moves only a machine that lacks a capability this app uses (limited), or one the app refused. A machine that serves all of it is left alone, whatever its build. Update Machine still moves any machine on a click.
  • What counts as an answer. A success must exit 0 and carry updated, plus the build it reached when it updated. An update that reports another build than the one it was going to is a failure. machine_update_unresolved (the CLI lost the machine before the outcome) and a CLI timeout are asked again with backoff, never recorded as a failure. Both toggles are read again just before the CLI runs.
  • No wait for idle, as Leo's words above ask: the machine updates whenever the queue asks, also while an agent works there, and the new daemon adopts the terminals and chats and resumes what it interrupted. It answers busy only when the CLI can't update now (machine_update_busy, machine_unreachable), and the queue asks again with backoff.
  • Not rebuilt from #24070: CloudMachineQuiet, its quiet read (person typing, attached UIs, cmux calls in flight, agents in a turn). With no idle wait nothing would read it. (Its client-rule bug, the daemon's own permanent kindless connection keeping a machine busy forever, goes with it.)
  • cloud.updateMachines (Debug Settings, next to agent.updatePeers, on): off leaves Cloud machines to Update Machine. CloudMoveCLI reads the answer's build and takes toBundled.

Timers and their stop conditions

None of its own: the queue's backoff nap and drain loop (#25387) run it, and link machine update keeps its existing deadline.

Tests

CloudMachineUpdateTargetTests (a fake machine and policy, a recording clock):

  • aBusyMachineIsUpdatedAtOnce (the default: no wait, no backoff); aMachineTheCLICannotUpdateNowIsAskedAgain (busy from the CLI, 2 minutes, then updated);
  • anIncompatibleMachineUpdatesRightAway, aMachineThatHasNotAnsweredIsLeftAlone, aFailureShowsOnceAndIsNotRetriedInALoop, theToggleOffMeansNoAutomaticUpdate, cliAnswersMapToOutcomes, aCLIErrorIsAFailure.
  • Review fixes: aMachineThatServesEverythingIsLeftAlone, aToggleTurnedOffBeforeTheCLIRunsStopsTheUpdate, aCLITimeoutIsAskedAgain, anUpdateToAnotherBuildIsAFailure, aSuccessLineWithoutItsOutcomeIsNoAnswer, and machine_update_unresolved ā†’ busy in cliAnswersMapToOutcomes.
  • All are new (the target is new), so they fail on base.

Review (Codex gpt-6-astra xhigh): the first round blocked on 2 P1s (a move back; unresolved recorded as failed) and 2 P2s (a toggle race; a malformed success accepted). All four are fixed in f3ea03e2, and the next round gave SHIP there. Later restacks onto #25394 are rebase only (same patch-ids).

Observed / Not run

No capture: no visible UI.

Agent session: big-red
šŸ¤– Generated with Claude Code