cmux next: computer use works on macOS 26: setup asks in the chat, the helper runs from a verified copy, restarts itself and acts fast

LU
10d ago
+8520-4007

On macOS 26, computer use in cmux NEXT could not work. Seen on 2026-10-27 on macOS 26.6:

  • An agent that called a computer use tool before setup waited 55 s with nothing on screen, then failed. The person never saw a question.
  • Screen Recording never became ready. The helper ran from inside the app bundle, so macOS checked its capture grant as cmux NIGHTLY, not as "cmux Computer Use". The tccd log showed the same pid, 13 ms apart: Accessibility as com.cmuxterm.cua (allowed), ScreenCapture as com.cmuxterm.app.nightly (denied).
  • Even with the grant on, the helper's capture check stayed false until the helper itself asked once for screen capture. cmux's Allow only opened System Settings.
  • "Quit & Reopen" in System Settings quits the helper. The app did not notice: it kept saying "ready", and every call failed until the app restarted.
  • Setup could hang on "Starting the cmux Computer Use helper…" for ever.
  • The Agent Activity page read the wrong socket, so it was empty and its Stop did nothing.
  • Tagged DEV builds gave agents no computer use tools at all.
  • The first click of a run took 3.3 to 3.6 s, later clicks 0.2 to 0.9 s each.

This PR fixes all of these, so an agent can set up and use computer use on macOS 26 from the chat.

  • Setup asks in the chat (cmux-tui/crates/cmux-tui/src/cli/mcp/computer_use_gate.rs, upstream.rs, mcp.rs): before a cua_* call (not check_permissions), cmux mcp serve reads the app's computer_use.status. Ready: the call runs. Off or a grant missing: it shows the existing "Allow computer use?" card with permissions.request {kind: computer_use}, waits up to 5 min (or the time a cmux_exec script has left), then runs the same call. Repeated calls join one card. The card names the app the call names (app_name, name, app, bundle_id) when that reads like an app name, else "It wants to use apps on this Mac."
  • Plain answers when it cannot run: denied, no answer, policy, no signed helper, version mismatch. "No signed helper" is now only that: a helper that could not start is failed, not unavailable. A starting helper, the helper v2 driver and an app that does not answer keep today's path. The check runs with the call, never on the server's dispatch thread.
  • One agent token per app launch (ComputerUseHelperDaemon.swift): every helper start serves it, and the endpoint (socket and agent token) is published from launch while the legacy driver is chosen. A chat started before setup, or before a helper restart, reaches the helper afterwards. When the token is replaced (a failed running check), the app sends the new endpoint to its running acpmux at once (AcpmuxHost.computerUseChanged), so new chats get it without a pane reconnect. The host token stays one per start and is never given to acpmux or an agent, but it is in the helper's environment (see Review, P2-2).
  • The helper runs from a verified copy outside the app (ComputerUseHelperStaging.swift): copied to ~/Library/Application Support/cmux/cmux-cua/helper/next-<scope>/ (or next-v2-<scope>/ for the helper v2), quarantine removed, signature checked, moved in with an atomic rename. A matching copy is reused; a changed or tampered copy is replaced. Every folder from home down must be a real directory of this user, not group or world writable. After launch the running pid is checked against the requirement captured before the copy. A mismatch stops it before the app or a chat uses it, and the launch's agent token is replaced, because that process got it in its environment. The setup's "drag this app" tile names only a verified copy.
  • Allow asks the helper first (ComputerUseSetup.open): with the legacy helper running, Allow for Screen Recording sends request_system_permission {name: screen_recording} over its socket with the host token, so the helper raises the one macOS capture prompt itself. Then System Settings opens as before. The agent card's Continue (SystemPermissionOpener) goes through the same path. Accessibility sends nothing: the toggle is enough.
  • The helper restarts when it exits (ComputerUseHelperExit.swift): a kernel process-exit watch through the launcher (no polling). On an exit the app did not cause, the state and tokens clear and the helper starts again, at most 3 times a minute; then the state is .failed. A stop, restart or quit cancels the watch first. The helper v2 gets the same watch, and also reaps the child and removes its endpoint.json.
  • Setup fails instead of hanging: after the status reads run out, or when the helper cannot start (copy, scope lock, launch or running check), the phase is failed, with Restart Helper in the Settings card, the onboarding step and the palette. Any recheck starts a new read budget. cmux setup computer-use names the restart as the next step. When the helper's pid changes, the setup reads again from "starting".
  • One helper per scope (ComputerUseHelperScope.swift, WorkspaceHelperLauncher.swift): a scope lock per socket, shared pending starts, and an older signed cmux-cua serve on the exact same socket is stopped first (5 s, off the main actor). Other tags, NIGHTLY and classic cmux have other sockets and are never touched.
  • Agent Activity reads the running helper (AgentActivityPageService.swift, AgentActivitySocketSource.swift): its socket and tokens come from the daemon and follow a restart. Export, open agent, open target and stop now report an error instead of failing silently.
  • DEV builds get the cua tools (cmux-tui/crates/acpmux/src/agent_tools/cua_helper.rs): only in a DEV app, acpmux looks for the installed NIGHTLY, RC or release helper in the same places as the app, and uses it only if it passes the Developer ID requirement (codesign --verify -R, 3 s timeout, fails closed, cached per binary). Release builds are unchanged.
  • Fast cursor: the helper starts with --glide-ms 150 --dwell-ms 30 when its manifest lists serve.motion or its binary_version is 0.8.12 or later (the pinned helper takes the flags but does not list them yet). An older helper gets no motion flags, so it does not exit at launch on an unknown flag. The cmux-cua skill tells agents to batch launch, snapshot and clicks in one script.

Security: the agent token reaches the same places as before and no longer rotates within one app launch. An agent still cannot turn Computer Use on or get a grant without the person's Continue. A helper copy that fails its checks is never launched. A folder on the copy's path with group write (for example ~/Library/Application Support/cmux made under umask 002, mode 0775) is still refused: its group is staff, which holds every local user. The log names the folder, owner and mode, and Setup shows Restart Helper. Not fixed here: the host token is in the helper's environment, so a process of the same user can read it with ps -E (older than this PR, P2-2 below).

Known gap: when the agent's call goes away (cancel, chat closed), its card stays until it times out.

Replaces #25795, #25831, #25832, #25837, #25865, #25938, #25901.

Changelog

  • Fixed: on macOS 26, computer use works: cmux asks for setup in the chat when an agent needs it, and the agent goes on in the same chat.
  • Fixed: computer use can take screenshots after Screen Recording is allowed.
  • Fixed: computer use starts again by itself after its helper quits, for example after "Quit & Reopen" in System Settings.
  • Fixed: Computer Use setup no longer hangs on "Starting"; it offers Restart Helper.
  • Fixed: the Agent Activity page shows computer use sessions and its Stop works.
  • Fixed: agents in development builds get the computer use tools.
  • Changed: the computer use cursor moves faster.

Tests

  • cmux-tui/crates/cmux-tui/src/cli/mcp/upstream_tests.rs (against a real child MCP server): a call before setup shows the card with the plain reason, then runs; a ready setup runs with no card; in the stdio server the setup read and the card wait run on the call's thread, not the dispatch thread; deny, timeout and unavailable answer without running the helper; a script that used 2 s of 120 s offers at most 117 s; another server never reads setup.
  • cmux-tui/crates/cmux-tui/src/cli/mcp/computer_use_gate.rs (unit): the status tool and an unknown setup run as before; a starting helper and the helper v2 are left to their own wait; phases the card cannot fix answer at once; failed is treated like starting; the wait is bounded by the call's limit.
  • cmux-tui/crates/acpmux/src/agent_tools/cua_helper_tests.rs: a release app gets nothing; a DEV app falls back; order NIGHTLY, RC, release; a candidate that fails the signature check is skipped.
  • ComputerUseEndpointTests.swift: the endpoint is published from launch; a chat started while off reaches the helper that starts later; a restart serves the same agent token; policy and the helper v2 publish nothing; quit withdraws it.
  • ComputerUseHelperStagingTests.swift: the copy is made outside the app, reused, replaced on change or tamper; a failed signature, a symlinked copy, folder or ancestor, and a writable or foreign ancestor are refused; a concurrent winner is used only when verified; old staging folders are swept; the setup tile names only a verified copy; the helper launches from its copy; a swapped running helper is stopped, is failed, gets no configuration, and its new agent token is pushed to acpmux; only a missing signed helper is unavailable (a launch with no pid, a group-writable ancestor and an unverified copy are failed); the motion flags go only to a helper that lists them; a swapped helper v2 gets no configuration or endpoint.
  • ComputerUseHelperExitTests.swift: an exit the app did not cause starts the helper again (legacy and v2); a stop, restart or quit starts no extra helper; an exit loop ends in .failed until Restart Helper; restarts older than a minute no longer count; the real kernel watch on a child process. These tests now use a passing staging fake (ComputerUseHelperPassingStaging.swift), the same for the endpoint and lifecycle tests, because their helper bundle does not exist.
  • ComputerUseSetupTests.swift: no answer gives failed after six reads; each recheck starts a new budget; restart shows progress then reads the new helper; a pending restart does not run after policy turns Computer Use off; Allow asks the running helper for the grant, and only opens System Settings when off; another helper reads from "starting"; a helper that keeps exiting shows failed with Restart Helper.
  • ComputerUseHelperLifecycleTests.swift, ComputerUseHelperScopeTests.swift, ComputerUseActivitySourceTests.swift: shared pending launches and quit during launch; one owner per socket scope, only a signed helper on the exact socket is stopped, an exited candidate counts as stopped; the activity page follows the helper's socket and tokens, unsupported operations report an error.
  • ComputerUseCursorSpeedTests.swift: the helper gets --glide-ms between 50 and 200 and --dwell-ms at most 50; without serve.motion it gets neither; the capability comes from the list or a binary_version from 0.8.12.
  • ComputerUsePermissionCardTests.swift: Continue on a computer use card with Screen Recording missing sends request_system_permission once and opens the Screen Recording list; with Accessibility missing it sends nothing.
  • AcpmuxComputerUseRefreshTests.swift: a changed endpoint reaches the running daemon, and a withdrawn one clears it; nothing is sent with no daemon, an attached daemon or an unsigned DEV daemon.
  • webviews/src/pages/settings/computerUseCard.test.tsx: the card shows the failed state and the restart action.

Observed / Not run

  • Ran locally (Xcode 26.6) on the head: swift test --package-path Packages/macOS/CmuxNext --filter 'ComputerUse|AcpmuxComputerUseRefreshTests|AgentPermission|AcpmuxLaunchSecurityTests|AcpmuxAttachTests|AcpmuxOfferToolsTests|AgentActivity|PermissionRequestPresenter': 121 tests in 26 suites passed.
  • Review fixes, red then green: the new tests were committed first (71ced84fdd3, with inert seams so they compile). On that commit the same filter failed 17 issues: the card tests (no request_system_permission, System Settings opened outside the setup), the endpoint push (nothing enrolled), the motion gate (flags always passed) and the failed-state mapping (.unavailable). On the fix commit all pass.
  • The two rewritten Rust tests were checked against a mutation: with the proxied call run on the dispatch thread, the_setup_read_runs_off_the_dispatch_thread fails; with the script's whole limit instead of its time left, a_script_waits_only_for_its_time_left fails (120 s). Both pass on the head.
  • cargo test -p cmux-tui --lib mcp: 119 passed. cargo test -p acpmux --lib agent_tools: 12 passed and cargo test -p acpmux --test agent_tools: 1 passed (before the review fixes; acpmux is unchanged since). cargo fmt --check: clean.
  • bun test src/pages/settings/computerUseCard.test.tsx: 4 passed. bun run check, step by step: all pass except check:type-scale, vp fmt --check, vp lint and acpmux-protocol:check, which fail the same way on base 2c480abe5cf in files this PR does not change.
  • python3 scripts/cmux-next/prepush-gate.py: no new violations. The push used CMUX_WEB_LINT_SKIP_REASON for those base reds only.

Seen live by Lucas in the tagged app cuall (all four PRs of this stack together), on macOS 26.6, before the review fixes:

  • An agent ran Calculator through computer use.
  • The first click of a run went from 3.6 s to 0.3 s.
  • The helper was killed, and the app started a new one in 1 s.

Not run: a real "Quit & Reopen" click in System Settings, Codex and other harnesses, the hosted verify-cmux-tui-hosted.sh run, a tagged build with the review fixes, a real acpmux enrollment after a token change (tested with a recording enroll).

Review

Fixed from the review of b98798af97d, each with a test that failed before the fix:

  • P1-1: the card's Continue now opens the helper's grants through ComputerUseSetup.open, so Screen Recording sends request_system_permission once (legacy driver, helper running); Accessibility sends nothing. ComputerUsePermissionCardTests.
  • P2-1: a changed endpoint (replaced agent token, published or withdrawn) goes to the running acpmux at once through _acpmux/computer_use_enroll, after any lookup in flight. Chats already running keep the token their computer use server started with; an unsigned DEV acpmux keeps its spawn endpoint. ComputerUseHelperStagingTests, AcpmuxComputerUseRefreshTests.
  • P2-3: --glide-ms and --dwell-ms go only to a helper that lists serve.motion or reports binary_version 0.8.12 or later. ComputerUseCursorSpeedTests, ComputerUseHelperStagingTests.
  • P2-6 (part): copy, scope lock, launch and running check failures are failed (Restart Helper; the gate leaves the call to the helper) instead of unavailable ("no signed helper in this build; do not call"). A group-writable folder on the copy's path stays refused, now logged with the folder, owner and mode. ComputerUseHelperStagingTests.
  • Tests that claimed too much: the "off the dispatch thread" check now runs the stdio server and compares threads; the script wait now asserts the bound from the time left, not <= 300.

Open, not in this PR:

  • P2-2: the agent and host tokens are in the helper's environment, readable by any process of the same user with ps -E. Older than this PR. Follow-up: hand them over a pipe or a 0600 file the helper reads and unlinks.
  • P2-4: the running check is by pid (an exec swap passes it), and the manifest runs the staged copy before any process check. Low impact while P2-2 is open.
  • P2-5: a joined card or a script can run the call after a transport timeout, and the status read and slack are not bounded by the script's time left.
  • P2-6 (rest): failed still runs the call (a 55 s helper wait) instead of its own refusal that names Restart Helper.
  • P2-7: acpmux runs codesign on a runtime thread in DEV builds, and its cache keys on mtime only.
  • P2-8: the Agent Activity Watch button always reports an error.
  • P2-9: stopAndWait blocks a pool thread, staging runs before the scope lock, and prepareRestart can drop a restart.
  • The pinned helper should list serve.motion in its manifest (cmux-cua repo); the version floor then becomes unneeded.