acpmux: the guest takes its caller's VM boundary, per session (vmBoundary on session/new)

Merged
TE
9d ago
+767-213

The guest's side of the VM boundary. On a Linux Cloud VM, a remote chain's Claude Code runs without bwrap only when the caller asked for that, for that one session, over its peer link. The guest never decides it.

The design, as agreed with the area owner: "the kind comes from outside the guest. The client side (the app or the daemon that starts the remote session) asks the Cloud API which kind the VM is, for the signed-in account, and passes that to the remote start; the guest never decides." And: "Agents keep sudo in Cloud VMs".

This is PR 2b, on top of #26309 (2a, the caller's side, merged as f4946bb4d65). The order:

  • Done: fail-closed #26227 (every Cloud VM sandboxes).
  • Done: web/ attach-endpoint vmKind #26222 (live).
  • Done: #26309 (2a), the caller keeps and forwards its opt-out.
  • This PR (2b): the guest takes it.
  • (3): the Cloud link passes --vm-kind owner into the lease.

What changes

  • Caller side: see #26309. It keeps the opt-out (saved peer or lease) and forwards _meta.acpmux.vmBoundary: true only for the person's own clients. A request forwarded for a paired machine says via: device.

  • Guest, accepting: the guest takes the field only from a connection that presented its peer token and asks for itself (ConnAuth::PeerToken and Control::Peer, not via: web), and only for the exact value true. It records SessionMeta.vmBoundary. A fork, sub-session or side chat keeps it, as they keep remote_origin. A session import (transfer) and a handoff target never carry it, so they stay sandboxed.

  • Guest, at each spawn: the guest honours the flag only while it takes the boundary: Linux only, and never once its own config says vmHost: false. It can tighten but never loosen. No marker, /srv/team or vmHost: true is read. The unsandboxed run keeps every other piece of the remote hardening:

    • the remote settings with disableBypassPermissionsMode;
    • --strict-mcp-config;
    • the refused-env scrub, plus CLAUDE_CODE_SUBPROCESS_ENV_SCRUB.

    It marks the session unsandboxed, so a Web client never controls it (remote.unsandboxed_agent).

  • Approve-all keeps vmHost's old rule, with the caller's opt-out in place of the guest's own say (vm_boundary_peer_approves, which replaces vm_host_peer_approves). It applies only when all of these hold:

    • the request comes from the owner's peer daemon (Control::Peer);
    • the session asked for no policy;
    • the machine is not a Mac;
    • MDM does not force full access off.

    MDM is checked again at every permission answer (policy_for). Nothing is widened.

  • Only the owner controls a boundary session. A paired machine's client credential is served as a peer (ConnAuth::Device, Control::Peer), and a peer may forward for one (via: device). Either is held to the Web's rules for a session at the boundary, and the Web's rules never prompt, steer, answer for, fork or switch the mode of one (unsandboxed_refusal, and copies in remote_guard).

  • Only reads from anyone else. A request that isn't the owner's and names a boundary session is refused, unless it's one of the read-only role's reads (Hub::refuse_at_vm_boundary). Methods added later are refused too.

    • The check runs at the top of handle_request and again right before the request is served, after every await in between. The session/cancel notification gets the same two checks.
    • The session it resolved is pinned into the request (sessionId), so a rename in between can't point the request at another session.
    • An ambiguous key (a prefix of more than one session) is refused, not passed on.
    • A key that names no session is let through: a boundary session could only get that name from the owner or its own agent.
    • _acpmux/warm skips boundary sessions unless the owner's own client asked.
    • mux/shutdown is refused for anyone but the owner while any boundary session runs.

  • _acpmux/vm_host toggle: off writes vmHost: false, and on removes the key (it is no longer refused). Only the owner's own clients may turn it on. Its vmHost reading means "this machine takes its owner's boundary". It is display only, and no marker feeds it.

Why a guest-side forger gains nothing

Nothing the guest writes changes another caller's session. A root process in the guest can read run/peer.token and send vmBoundary: true itself. All it gets is an unsandboxed Claude that it started, as the agent user, which its own root could already run.

What no check inside the guest can do is hold a sandbox against root in that guest. That limit is the "the VM is the boundary" residual.

Risk

Medium: this re-opens an unsandboxed path. It is gated on a flag that only a peer-token caller can send, and that the caller sets only from its own config or a Cloud-API-attested lease.

Session meta now persists vmBoundary, like remote_origin. Root in the guest could edit it, which is the same residual as above.

Dispatch (_acpmux/dispatch), handoff targets and imports never carry the flag, so they stay sandboxed (fail closed).

What a Cloud VM owner sees:

  • Before PR 3: no change. Remote Claude stays in bwrap.
  • After PR 3: remote chains from their own app's link run unsandboxed.
  • A team VM: the API says team, so it stays sandboxed.

Fleet lane VMs with vmHost: true in their own config.json stay sandboxed. Only the caller's vmBoundary can loosen.

Size: about 1,000 changed lines, of which about 450 are tests. That is over the ~800 guide because of four review rounds' fixes, each with its test. The caller's half already landed as #26309.

Tests

These fail on base (b727b489) and pass after:

  • tests/remote_sandbox.rs only_the_callers_vm_boundary_over_its_peer_link_skips_the_linux_sandbox (the host has no bwrap):
    • Only a Peer connection with vmBoundary: true runs, with or without vmHost: true.
    • It records remote_sandbox {vmBoundary: true}, and its argv keeps --settings, --strict-mcp-config and disableBypassPermissionsMode.
    • A Web prompt to it is refused with remote.unsandboxed_agent.
    • These stay refused with "bwrap unavailable": the field from a Web connection, the field absent, the string "true", and vmHost: false.
  • the_callers_vm_boundary_starts_a_peer_session_with_no_policy_approve_all:
    • The boundary with no policy gives approve-all.
    • An explicit ask stays ask.
    • No boundary, or vmHost: false, gives ask.
  • only_a_vm_boundary_lease_starts_the_peers_sessions_at_the_vm_boundary: two hubs.
    • A lease with vmBoundary makes the VM record it for the Mac's own client's session.
    • It does not for a Web client's session through the same lease.
    • A plain lease, even with the client's own vmBoundary: true, does not.
  • a_paired_machine_or_web_client_only_reads_a_session_at_the_vm_boundary:
    • The owner's prompt runs.
    • A paired client credential's prompt, and a prompt a peer forwarded for one (via: device), are refused with remote.unsandboxed_agent.
    • The paired machine's fork is refused, and no session is made.
    • Its vm_host {on: true} is refused, while the owner's is not.
    • A paired machine's and a Web client's session/set_model are refused, and so is a paired machine's session/cancel.
    • _acpmux/warm starts the session only for the owner.
    • A paired machine's _acpmux/events read still answers.
  • a_non_owner_request_keeps_its_session_and_never_stops_the_boundary:
    • A paired machine's request stays pinned to the session it named, across a rename.
    • An ambiguous prefix is refused.
    • A paired machine's own session/new with a matching name still starts.
    • A paired machine's mux/shutdown is refused.
  • tests/vm_host.rs:
    • The toggle goes off and on again with no restart.
    • The marker and vmHost: true decide nothing.

Also added: unit tests for asked, owner and the approve-all rule (including MDM). The existing peer_status, vm_host, peer_lease, web_token_exposure and transfer_cli suites still pass.

Before and after

Base b727b4892aa with only this PR's sandbox tests in tests/remote_sandbox.rs (Linux, cmux-ci):

test a_guest_written_owner_marker_or_vm_host_never_skips_the_linux_sandbox ... FAILED
test only_the_callers_vm_boundary_over_its_peer_link_skips_the_linux_sandbox ... FAILED
test the_callers_vm_boundary_starts_a_peer_session_with_no_policy_approve_all ... FAILED
test only_a_vm_boundary_lease_starts_the_peers_sessions_at_the_vm_boundary ... FAILED
vmb-peer: {"error":{"code":-32000,"data":{"reason":"remote.sandbox_refused"},"message":"remote chain refused: bwrap unavailable: ..."}}
vmb-approve: left: String("ask") right: "approve-all"
lease-vmb: own left: false right: true
test result: FAILED. 5 passed; 4 failed

(The first test changes only the vmHost display. On base it reads false everywhere; here it reads "takes the boundary", which is true on Linux.)

The paired-machine test, on this PR's earlier heads:

before the owner check:      test a_paired_machine_or_web_client_only_reads_a_session_at_the_vm_boundary ... FAILED
                             device prompt: {"id":1,"jsonrpc":"2.0","result":{... "stopReason": ...   (the paired machine's prompt ran)
before refuse_at_vm_boundary: ... FAILED   device model: {"id":3,"jsonrpc":"2.0","result":{}}   (its model change went through)

The pin and ambiguity test, on this PR's earlier heads:

before the pin:        test a_non_owner_request_keeps_its_session_and_never_stops_the_boundary ... FAILED
                       not pinned: {"modelId":"other","session":"plain"}
before the ambiguity:  test a_non_owner_request_keeps_its_session_and_never_stops_the_boundary ... FAILED
                       an ambiguous key was admitted: {"modelId":"other","session":"pla"}

This head, on the tip (cmux-ci, Linux):

cargo fmt -p acpmux --check: ok
cargo clippy -p acpmux --all-targets -- -D warnings: ok
lib vm_boundary vm_host peer_status peer_lease peers remote_sandbox transfer handoff catalog_reload web_control remote_guard pairing trust_gate warm: passed
--test remote_sandbox 11, vm_host 5, peer_lease 8, web_token_exposure 6, transfer_cli 1, pairing 7, conn_auth 3,
  remote_guard_codex_bridge 13, peer_integration 50, remote_new_hints 8, peer_origin 4, local_app_origin 7, hub_integration 510: all passed

The full cargo test -p acpmux fails on the tip the same way it fails here: agent_tools, codex_auth_broker, daemon_chat_open, ephemeral_run, hermetic_daemon_guard and session_pool.

Agent session: agent session
šŸ¤– Generated with Claude Code