acpmux: fail closed: no guest-writable vm-kind or vmHost turns off the remote sandbox

Merged
TE
10d ago
+226-174

The sandbox decision for this area (2026-10-27), verbatim:

"Fail closed now. Nothing the guest can write may turn off the sandbox. Change remote_sandbox.rs so the vmHost exception no longer reads /etc/cmux/vm-kind (or anything else on the guest); until an attested source exists, every Cloud VM is treated as a team VM and remote Claude runs inside bwrap with the probe and canary. Small PR, a test that writes "owner" into the marker and still gets the sandbox, Codex xhigh review, land."

Takes back #25749: its vmHost default read the guest-writable /etc/cmux/vm-kind (and config.json) to run remote Claude unsandboxed and approve-all; that trust input is removed until an attested source exists.

Why

On a Cloud VM the agent user (cmux) has NOPASSWD:ALL sudo (web/services/vms/images/devbox/Dockerfile:100, images/workUser.ts:52). An agent could write owner into /etc/cmux/vm-kind (and remove /srv/team), or write vmHost: true into acpmux's own config.json. Any of those made Hub::vm_host() true. Off macOS, that ran a remote chain's Claude Code with no sandbox at all, and the owner's peer daemon's new sessions started approve-all.

What changes

  • hub/remote_sandbox.rs: the vmHost exception is gone. A remote chain's Claude Code always goes through sandboxed_claude_plan (the bwrap probe, then the canary) on Linux, the same as on any other host. Seatbelt on macOS was already unconditional.
  • hub/vm_host.rs: Hub::vm_host() returns false until an attested source exists. Nothing on the machine decides it: not the marker, not /srv/team, not config.json. So vm_host_peer_approves never gives approve-all either. The marker (cmuxVm) and config.json's value (configured) are still read and shown by _acpmux/vm_host, but they decide nothing. on is refused with a new reason, vmHost.unattested. off still saves vmHost: false.
  • agent/command.rs: REMOTE_ENV_POLICY_ID goes from remote-env-v3 to remote-env-v4. When the new daemon adopts a remote-origin agent host that an older daemon started, it ends that host instead of reusing it (hosts.rs adopt_one), so a host that vmHost left unsandboxed does not outlive the upgrade. Every remote-origin host restarts once on its next prompt. Local sessions, including lane-place lanes, are not affected.
  • CLI line (acpmux host vm-host), README, config docs and docs/acpmux-schema.json now say this.
  • Not touched: cmux-vm-kind-sync, role.rs record_owner_kind and the web/ marker writers (#26179/#26182 own them). The attested-kind work comes in later PRs; this one is self-contained.

What a Cloud VM owner sees now

  • A Claude chat started on the VM from the Mac (a remote chain) runs inside bwrap after the probe and canary. It can write only its session folder and its temp folder. It cannot write .git/hooks or CLAUDE.md or reach loopback, and its network goes through the policy proxy. The Cloud image already installs bubblewrap (Dockerfile:76). If bwrap cannot run there, the chat is refused with remote chain refused: bwrap unavailable: … (remote.sandbox_refused). It never falls back to running unsandboxed.
  • A chat the owner's peer daemon starts with no policy starts in ask, not approve-all.
  • acpmux host vm-host prints vmHost off (a VM cmux created: sandboxed until an attested source says it is a dedicated agent VM).
  • In Settings > Machines, the "Dedicated agent machine" switch reads off. Turning it on shows vmHost stays off here: no attested source says this is a dedicated agent VM, …. Its help text ("Agents here run without the sandbox…") is now stale. That is Swift copy, left for the attested-kind follow-up, which brings the switch back.
  • A Claude host started unsandboxed by an older daemon has no canary record. When this daemon adopts it, hosts.rs marks it unsandboxed and refuses Web control of it (remote.unsandboxed_agent) until a fresh, sandboxed spawn.

I did not test this on a real Cloud VM: there is no Cloud VM peer I could reach read-only. The Cloud VM behaviour above is reasoned from the code and the image's Dockerfile. I did check a GCP lane VM (also a vmHost host) read-only: bubblewrap 0.9.0 is installed, and bwrap --unshare-all --ro-bind / / true succeeds as the agent user.

Risk

Medium. This is a deliberate behaviour change on every machine that had vmHost on:

  • On Cloud VMs: remote Claude now runs sandboxed and asks first, as described above.
  • On the fleet's GCP agent_vm hosts, where gcp-session-host vm-host writes vmHost: true: Claude sessions started over the peer link (acpmux new --host, the app) now run in bwrap and ask first. Lanes started by lane-place are not affected: it runs acpmux new on the host's own unix socket, so they are local sessions and not remote chains. Codex sessions are not affected (the remote chain sandbox is for Claude Code).
  • Failure mode: a host where bwrap cannot run refuses remote Claude with the reason. It never runs it unsandboxed.

Tests

Both the before and after runs are cmux-ci test on 32 vCPU Linux with bubblewrap installed and ACPMUX_REQUIRE_BWRAP=1.

  • tests/remote_sandbox.rs a_guest_written_owner_marker_or_vm_host_never_skips_the_linux_sandbox covers three cases: an owner marker, config vmHost: true, and both. Each time a remote Claude spawn still goes through the bwrap probe: with bwrap missing it is refused, never run.
  • tests/support/remote_sandbox_linux_stack.rs linux_remote_chain_with_vm_host_true_still_runs_in_the_production_sandbox is the real daemon binary with vmHost: true. The canary passes (a remote_sandbox {canary: passed} record, no vmHost record), the agent's writes outside its folders and its loopback connections are denied, and it inherits none of the daemon's secrets. Both production-spawn tests now check the canary record and the environment.
  • tests/vm_host.rs: an owner marker or config vmHost: true is shown (cmuxVm, configured) but vmHost stays false, and on is refused with vmHost.unattested.
  • Two in-process Linux tests only ran Claude because vmHost turned the sandbox off: a_remote_claude_gets_only_the_selected_login_credential and a_remote_chain_inherits_only_profile_and_safe_base_environment. They are now macOS-only, because the in-process Linux spawn goes through the keeper, which needs the acpmux binary. The production-spawn test above checks the Linux environment instead.

Before (base code + these tests, cmux-ci test run 40523151443)

test a_guest_written_owner_marker_or_vm_host_never_skips_the_linux_sandbox ... FAILED
  panicked at tests/remote_sandbox.rs:281: owner-marker: remote Claude ran without the sandbox
test linux_stack::linux_remote_chain_with_vm_host_true_still_runs_in_the_production_sandbox ... FAILED
  panicked at tests/support/remote_sandbox_linux_stack.rs:156:
  {"claude":"allowed","cwd":"allowed","git":"allowed","loopback":"allowed","outside":"allowed","proxy":"missing"}
test linux_stack::linux_remote_chain_uses_the_production_sandbox_without_a_switch ... ok
test result: FAILED. 4 passed; 2 failed
tests/vm_host.rs: an_owner_marker_is_shown_but_never_turns_vm_host_on ... FAILED
  left: {"cmuxVm": true, "configured": null, "vmHost": true}  right: {..., "vmHost": false}
  (and the 5 other vm_host tests: vmHost true from the marker or config, reason vmHost.not_cmux_vm)
test result: FAILED. 0 passed; 6 failed

After (this branch, runs 40523107994 and 40524969857)

tests/remote_sandbox.rs: test result: ok. 6 passed; 0 failed   (20 of 20 repeated runs ok)
tests/vm_host.rs:        test result: ok. 6 passed; 0 failed
--lib vm_host peer_status: test result: ok. 11 passed
cargo clippy --locked -p acpmux --all-targets -- -D warnings: clean

In the first after-run, the new vmHost-true production-spawn test was refused once at session/new. I did not capture the message. 35 runs of it since then (15 of the stack tests alone, 20 of the whole binary) all pass.

Out of scope (reported, not changed here)

  • cmux-tui/src/agent_start_host.rs also reads /etc/cmux/vm-kind: an owner marker lets the app start local-control chats on the VM. That route is not a remote chain. Its chats were never sandboxed, before vmHost or after, and its own docs say it is not containment. It belongs to the attested-kind follow-up.

  • Peer sessions that got vmHost's implicit approve-all on an older daemon keep their saved policy. The owner's peer may ask for that policy explicitly (remote_guard.rs), and those sessions now respawn inside the sandbox. The store cannot tell an implicit grant from an explicit one, so a migration would be a separate change.

  • Reviewed by an independent Codex gpt-6-astra xhigh session: SHIP at 1c42c6a3fb78 (after two rounds; its findings were the pre-upgrade host reuse, fixed by the env policy v4 bump, and the two items above).

Agent session: agent session

🤖 Generated with Claude Code