#36614 cmux-rd host: the person drives their own XI2 pointer, so the agent's computer use keeps working while they hold control

What

On a NEXT Cloud machine, the person watching the agent display :99 through cmux.rd/1 and the agent's computer use now each drive their own pointer, so both can act at the same time.

• cmux-rd host's injector (cmux-tui/crates/cmux-rd-host/src/inject.rs) finds the XI2 master cmux person by name, or adds it, before the first injected event. It then makes that master its X connection's ClientPointer. The X server routes core XTest input through the sending client's ClientPointer: Xext/xtest.c calls PickPointer(client) / PickKeyboard(client) and then GetXTestDevice. So the person's clicks and keys go to the new master and its paired keyboard. cmux-cua 0.8.12 keeps sending core XTest on its own connections, and those still move the Virtual core pointer. cmux-cua does not change.
• Crash recovery. The X server does not release buttons or keys that a dead client held through XTest, but removing a device does (dix DisableDevice calls ReleaseButtonsAndKeys). So if a host finds a cmux person master that no session of its own holds, it treats it as a crashed host's: it removes that master and makes a new one. The last session in the process removes the master, and its slaves return to the Virtual core devices. Find-or-create and removal run under one per-display lock. If attaching fails it retries once, and a failed RemoveMaster is logged. A view-only session never injects, so it never adds the master. If XI2 masters are unavailable, the injector logs once and drives the core pointer as before.
• Hold guard. In a staging spike, Chrome dropped both clicks when one master pressed while the other held a button. So a person's press, or scroll, waits while the core (agent) pointer holds a button. The host checks with XIQueryPointer, and only for presses. The wait lasts at most 2 s, and then the press goes anyway. Every later event queues behind it, in order. The session loop pumps the queue every 20 ms while it waits. When control ends, the session loop releases the viewer's input and drops the queue.
• x11rb gains the xinput feature. No image unit changes are needed: the host already holds the display's X cookie.

Spike on a staging Cloud machine

Staging runs 42030597224 and 42031407018, plus three more on 2026-11-05.

• Pointers are independent. A second master on its own connection moves only its own pointer. The core pointer never moved during any of the agent's moves or drags.
• Alternating clicks land. Clicks from both sides, 30 ms apart in separate Chrome windows, landed 3 of 3 for each side.
• A held press collides. When one side holds a button and the other clicks, Chrome drops both clicks. The hold guard above prevents this.
• Keyboard: a known gap this PR does not change. On :99, with no window manager, typing into Chrome arrives from no one.
  • Plain core XTest keys fail even with no second master present.
  • cmux-cua's own type_text fails too. In foreground mode it reports "Typed 15 characters", but nothing reaches the field. In background mode it refuses with "no focus-free input backend".
  • The keymap is fine: keycodes 38-46 map to a-l.
  • So the failure is X focus or window activation without a window manager, not the second master.
  • Today the agent's browser use goes through CDP (cmux-browser-host), not X keys.

Hook for the same-window wait (follow-up)

Leo wants the agent to wait only when both act on the same window. While the person holds control, the host keeps the root window property _CMUX_RD_PERSON (type STRING, JSON):

{"v":1,"pid":1234,"pointer":11,"keyboard":12,"window":4194305,"button_held":false,"last_input_unix_ms":1762340000000}

• window is the toplevel (root child) under the person's pointer.
• button_held says the person holds a button, so the agent can also wait before its own press or drag.
• The host refreshes it on every press, key and scroll, at most every 200 ms on motion, and once more after motion settles.
• The host deletes it with the master. After a crash, last_input_unix_ms stops moving and pid is gone.

It lives on the display, which the agent already shares, instead of in a new file under /run/cmux-rd (#36416 makes that dir 0711). Any client on the display can read it. A follow-up in cmux-cua will check it before an acting call on that window and is not part of this PR. Faking the property only makes the agent wait on itself, so the property does not need to be tamper-proof.

Cursors

• The agent's cursor shows in the frame through cmux-cua's own overlay. That is an override-redirect window clipped with XShape, and it works on Xvfb without a compositor.
• The person's cursor is drawn by the viewer, because Xvfb puts no cursor in the captured picture.
• No CursorPos datagram work is in this PR.

Tests

• Unit tests: the XIAddMaster length, picking the named master pair, and the property JSON.

• Hold guard unit tests:

  • With nothing held, everything passes.
  • Motion never queries X.
  • A press and everything after it wait until the agent lets go, then go in order.
  • After 2 s the press goes anyway.
  • Scrolls wait like presses.
  • Ending control clears the queue.
  • When the queue overflows, everything goes.

• person_master_keeps_the_core_pointer_for_the_agent (ignored by default, needs Xvfb) runs on a real Xvfb:

  • The person clicks a window at (700, 500), and the core pointer stays where the "agent" put it.
  • The agent then moves the core pointer, and the person's master stays put.
  • The agent holds a button, and the person's press waits until the agent lets go.
  • The property names the window under the person.
  • A second injector reuses the master.
  • Ending the last session removes the master and the property.

• crash_reuse_releases_a_crashed_hosts_button (ignored by default, needs Xvfb): a helper process holds a button through the master and aborts. The button is still down. A new host then replaces the master, and no button stays held.

• The Xvfb tests start Xvfb with -displayfd, check that it is still alive, and kill it on drop, pass or fail.

  It ran on the fleet at 63e198d6: the whole crate with --include-ignored passed 29 of 29, fmt is clean, and clippy is clean for the changed files (run 42041428139).

README: --peer-uid 1000, because the receiver runs as the work user (as #36416 sets), and Take over now drives the person master.

Changelog

Changed: after Take over on a Cloud machine's desktop, your input drives its own pointer, so the agent's computer use keeps working beside you instead of fighting over one cursor.

Why

Leo, 2026-11-05:
"Also go check in on the state of affairs of screen sharing remotely on these machines, like the cloud machines, because
there was so much VNC work. But it's also like, I feel like it wasn't really linked up end to end here, and we have an
opportunity quite soon to start doing that. [...] And then it became AI computer use!!"

Leo, on #36080: "is that really necessary? can't you both do it at the same time?"

Leo: "don't stick it into dev. Put it into prod".

Agent session: agent session

🤖 Generated with Claude Code

---

^{Open in [code]smith · Need help on this PR? Tag @codesmith-bot with what you need.}