PR #36080 - manaflow-ai/cmux

Summary

When a person and an agent share one Mac, the agent now stops typing and clicking as soon as the person starts. In the Computer Use helper, a listen-only event tap tells the agent's input from the person's. The rule is ported from Petal's ai_chat/takeover.rs and control_gate.rs (kiruna-labs/petal @ 7ac436a, Apache-2.0, credited in each file).

Leo, 2026-11-05: "honestly i'd rather us look at lot more at Petal and stuff like we ought to be able to get a lot more efficient and start going towards crazy stuff... after being able to do the basics end to end and adding eval measurements, ofc", and on the Petal items: "obv do all these and more".

Who made an event.

The agent's: it carries cmux's marker (kCGEventSourceUserData = "CMUXSYNT") or comes from the helper's own pid. Cua Driver runs inside the helper, so every route it posts on carries that pid.
The person's: any other key down, click, scroll or drag. Pointer movement alone is not a takeover.

The rule (TakeoverRule, unit-tested):

The person takes over on any deliberate input of theirs.
An acting tool call (readOnlyHint false) waits until the person has been quiet for 1.5 s, then runs.
If the person keeps going for 10 s, the call is refused with person_in_control and does not run.
Observing calls (screenshots, window state) never wait.

Placement. The gate runs in CuaDriverRuntime.invoke before the machine-consent gate, so consent stays the last check before the driver acts.

Permissions. Neither grant is ever requested.

The tap starts only when the helper already has Accessibility (AXIsProcessTrusted, which never prompts), in line with #26123.
It listens to keys only with Input Monitoring already granted (CGPreflightListenEventAccess, which never prompts). Otherwise clicks, drags and scrolls still take over; typing alone does not.
Health means the tap is enabled and a marked, in-place mouseMoved came back with its marker.
Without that, the gate is open ("unwatched") and Computer Use works as it does today. Failing closed would turn Computer Use off on every Mac without the grant.

Measurement. cmux-cua-helper --takeover-bench, also run as the display-only TakeoverBenchProcessTests, does two things:

It posts harmless agent input (a zero scroll, an in-place move, F19) on each route Cua Driver uses, and counts false takeovers.
It times a second process's stamped input from post to tap callback.

The logic is Swift, not Rust: it has to run inline in the process that posts the input, just before the driver call, and the helper links no Rust of ours.

Prior art checked.

#26123 (permission prompts deferred) is not affected.
#34913 (Aziz, per-app approvals) already makes "Allow Once" one-shot and consumed by the attempt. A sticky per-session Deny is proposed there as a comment, not duplicated here.

Measurements

Not measured live yet. The display run (step ac1274193aa39251f8129d3f, a display mini's console session) built the bench and ran it, and it reported {"ax_trusted":false,"listen_access":false,"ok":false}. On the display minis, Accessibility belongs to the signed dev helper (com.cmuxterm.cua.dev, Apple Development), not to a binary built in a test step. The tap cannot start without that grant, and the bench refuses rather than report numbers it didn't measure.

Getting the numbers needs the helper re-signed with this code (build-cua-helper-dev.sh, then sign-cua-helper-dev.sh on the Mac with the identity), then cmux-cua-helper --takeover-bench run from that signed bundle on a display mini. Until then the rule is covered by the deterministic unit tests (TakeoverTests).

There is no "before" row: before this PR the helper had no takeover detection, so the agent kept acting over the person 100% of the time.

Testing

Check	Where	Head	Result
TakeoverTests, MachineConsentTests, CuaTargetGuardTests, HelperTargetScopeTests, BridgeConsentIntegrationTests, HelperSocketProcessTests	cmux-ci steps `a91b5f1291c4005d1b49af82` / `cu-codrive-tests-762cc71`	491e570f, 762cc713	all 6 suites passed at both
TakeoverBenchProcessTests (display)	cmux-ci step `ac1274193aa39251f8129d3f`	762cc713	ran; could not start its tap (no Accessibility for a step-built binary)
prepush-gate	local, no build	each push	no new violations

Observed / Not run

Observed: the suites above, and the display bench running up to its grant check.
Not run: live takeover latency and false-takeover numbers, which need a signed helper (see Measurements).
Not run: a live agent session with a person typing at the same time, in a DEV build. DEV builds embed the signed, pinned helper (scripts/cmux-next/cua-helper-dev.pin.json, last rebuilt 2026-10-17), not the helper built from source. This change reaches DEV builds only when the pin is re-signed on a Mac with the Apple Development identity (build-cua-helper-dev.sh then sign-cua-helper-dev.sh). That is also true of every helper change since 10-17 (#25844, #27398, #33066).

Follow-ups

Report takeover: watched|unwatched|person in cua_status, and show "You're driving" on the preview card.
iPhone and live-preview telepointers: a separate PR, plus the WebRTC side by message.
Re-pin the dev helper automatically when its source tree changes.

Changelog

Added: Computer Use pauses the agent's clicks and typing while you use the Mac, and resumes when you stop.

risk:normal. No capture: the change has no UI; the bench numbers above are the evidence.

Agent session: agent session

🤖 Generated with Claude Code