feat(cloud): machines update their cmux-tui from a signed channel, no image bake

Merged
TE
1d ago
+2081-14

Leo, 2026-11-05: "literally all you're changing is cloud code. So presumably, I mean it's just the production deployment order. Like actually, like now that we're at it again, like can you think of ways to speed up this workflow?"

Until now a machine-side fix reached a Cloud machine only after a new image: pin cmux-tui, bake, approve the image, bump the API's image, create the machine. This PR makes a merge reach running machines by itself, minutes after its cmux-tui build is published. The image becomes a cache: a machine from a stale image boots its image's build and updates itself 30 s later.

CI publishes on merge. A new point-machine-channel job in cmux-tui-artifacts.yml runs after verify-publish on every feat-cmux-next push whose commit-addressed build is published. It writes https://files.cmux.com/cmux-tui/channels/cloud-machine.json, which holds the immutable URL, sha256 and size of cmux-tui and cmux-tui-hook for x86_64 and aarch64 musl. The sha256 values come from the commit manifest and are checked again against a download of each binary (scripts/ci/cmux-machine-channel.py). The file is signed with the web OTA ed25519 key (CMUX_WEB_OTA_SIGNING_KEY, artifacts environment; it never leaves CI) using scripts/cmux-next/web-ota-pointer.py, with the same envelope and the same newer rule, so the channel never moves back.

The two kinds of signed file cannot stand in for each other. The machine requires purpose: cmux-cloud-machine-update and channel: cloud-machine, and the app's web updater refuses any channel but its own. A test keeps the Rust key list equal to web-update-keys.json.

Machines update themselves. The new updater role of cmux host run (cmux-tui/crates/cmux-host/src/updater) checks the channel 30 s after start (boot, or an update's restart), on every resume and every 5 minutes:

  • It verifies the signature before it reads any field, refuses an expired file and never installs a build older than one it applied.
  • It waits until no acpmux session runs a turn or waits on a permission (acpmux's own --if-idle rule), and checks this again right before the flip.
  • It downloads both binaries into /opt/cmux/store/<sha256>/ (sha256 and size checked), builds the next profile as a copy of the current one with only the cmux-tui and hook links moved, and flips /opt/cmux/current with one rename.
  • It arms a guard timer that runs the previous build's cmux host update guard 20 minutes later, then restarts the work user's acpmux.service (agents keep running and are adopted), the other cmux-* services that run the old binary, and last cmux-host.service. Terminal hosts survive in their scopes.
  • The new cmux host run must show the session host and the acpmux daemon running the new store entry within 10 minutes. Then it records the update, disarms the guard and keeps 3 profiles plus the baked one; otherwise it flips back.
  • A build that never starts is rolled back by the guard. A rolled-back build is never tried again.

Off switches. /etc/cmux/no-self-update turns it off. The bake writes that file while its builder runs and removes it at park; the image smoke writes it on its clones, so a bake measures the snapshot as baked. A machine whose ~/.cmux/bin/cmux-tui the Mac app replaced in place (cmux link machine update) also leaves itself alone. Operator verbs: sudo cmux host update status|check.

Docs: docs/cloud-guest-upgrades.md has a new "Self-update" section.

Machines from images baked before this PR have no updater. They get it from their next image, or from one sudo <new cmux> host update check. The proof below used the second path.

🤖 Generated with Claude Code

Agent session: agent session


Autofix with [code]smith
Open in [code]smith · Need help on this PR? Tag @codesmith-bot with what you need.


Summary by cubic

Cloud machines now update their cmux-tui and cmux-tui-hook from a signed channel within minutes of a merge being published, instead of waiting for a new image bake. The image becomes a cache: a machine from a stale image boots its baked build and updates 30 s later.

  • CI publishes cloud-machine.json pointing at each commit whose build is published, signed with the web update key that never leaves CI; the channel is published with a compare-and-swap, so a slower run never moves it back.
  • The new updater role of cmux host run checks the channel at boot, on resume, and every five minutes; verifies the signature, waits for idle acpmux sessions (with a last check right before the flip), stages binaries into the store, flips /opt/cmux/current, restarts services, and rolls back builds that don't serve within ten minutes. A flock serializes checks so only one commit, rollback, or guard runs at a time; a stop and the flip are serialized too, so nothing flips or restarts after a stop returns. The guard timer uses --collect so the next update can recreate it.
  • The idle check fails closed: an unreadable acpmux state counts as busy, while a stopped daemon is idle. A stopped updater kills its download and changes nothing further.
  • /etc/cmux/no-self-update disables self-update; the bake writes it, and the smoke writes it with its first command on a clone, so the snapshot is measured as baked. Operator verbs are cmux host update status|check|guard.
  • Machines from images baked before this PR lack the updater; they get it from their next image or a manual cmux host update check.