cmux-tui: the person's key takes no client registry or session lock (R-DAEMON-STATE S1 step 1, behind CMUX_TUI_RDS_INPUT)

Merged
LA
3d ago
+778-27

Summary

The person's keys still wait in the daemon behind other clients. #31940 took the session lock (Mux::state) off the key, but a key line from the app's terminal view still takes the client registry lock (ClientRegistry.state) about nine times: the daemon handoff checks, the remote-client and remote-device checks, the request's actor, the kept terminal of the key and the reply's incognito filter. Every connect, disconnect, attach, detach and capability change of every client holds that lock, and an agent or CLI storm connects and disconnects many times a second. Under the L6 storm the build with #31940 still showed a daemon key part of p99 20-24 ms and max 74-223 ms (STATUS 7761).

This is stage S1 step 1 of the R-DAEMON-STATE rewrite (handoff/rewrites/R-DAEMON-STATE/DESIGN.md, section 2.6: the input path takes no shared lock; readers load immutable snapshots).

  • published.rs (new): Published<T, S>, a mutex-guarded value whose readers load an immutable projection with one atomic load (arc-swap). Writers serialize on the mutex as before. A guard that was borrowed mutably publishes the new projection when it drops, while it still holds the mutex, so projections are published in the order of the changes and no change can skip its publication.
  • ClientRegistry.state becomes Published<ClientRegistryState, ClientFacts>. ClientFacts holds the handoff state and, per connection, its transport, remote-device flag, base actor and which surfaces it can type into.
  • The session's three routing tables (surfaces, terminal_catalog, terminal_catalog_by_runtime) become copy-on-write (Arc<HashMap>). A state guard that changed one publishes a RouteSnapshot (three Arc clones) when it drops, under the lock; the snapshot it replaced is freed after the unlock. Mux::surface reads the snapshot with one atomic load.
  • With the stage switch CMUX_TUI_RDS_INPUT=1, the line and key paths take neither the registry lock nor the session lock: daemon_handoff_in_progress / daemon_handoff_committed, transport_of (so is_remote_client), is_remote_device (refuse_line, line_to_recheck, prune_reply), connection_actor, Mux::surface, and input_surface (the published routes give the terminal; the published facts say whether the client types into it, for the person's input QoS). On this path the key no longer uses the kept terminal of #31940 or its route generations, so there is no cache to invalidate. With the switch off (the default) every path is unchanged.
  • The switch is per mux (rds_switch::InputLaneSwitch, read once when the mux is built) and also decides whether anything is published: off, the registry builds no projection and the state keeps no route snapshot, so a route write never copies a shared table. State::clone (25 production sites) now shares the three route tables instead of deep-copying them, on both arms.
  • Failure rule: a poisoned registry still fails closed. Published::load gives the projection in the error, as Mutex::lock gives its guard; transport_of maps it to None (remote), the same as before.
  • Cost on the lane: one projection per registry change, O(connections) under the registry lock, bounded by the 64-connection cap; one copy of a route table the first time a guard writes it after a publication. Stage S2 makes both O(changed).
  • arc-swap 1.9 is already in the lockfile and in THIRD_PARTY_LICENSES.md (through iroh-dns); cmux-tui-core now names it.

Remote relay analysis (risk:high: the PR touches server/remote_relay/mod.rs): no policy changes. The same facts decide the same answers; only where they are read changes. A connection's transport is fixed at register, its remote-device flag is sticky, its base actor is fixed at accept and hello; each is written under the registry lock and published in the same critical section. A lock-free read linearizes before a change that is still in progress, as a reader that took the lock first did before. A frame of an unregistered or removed client still reads as remote (fail closed). No allowlist, no new method, no new effect, no local command or content execution, no access to unowned objects.

Routing analysis: a key on the lane goes to the terminal its id routes to in the last published state. A respawn, close or move publishes when its state guard drops, so a key in flight is linearized before or after that change, never into a half-applied one; a respawn under the same id sends the next key to the new shell (test below).

Owner: PL-daemon (session cmux-b6) owns cmux-tui/crates/cmux-tui-core/; told in STATUS.

Testing

Red then green with the same focused hosted command (./scripts/verify-cmux-tui-hosted.sh --filter rds_input_lane, cmux-tui/AGENTS.md):

Tests:

  • server::tests::rds_input_lane::a_key_waits_for_no_client_registry_lock: two keys through the real connection path (handle_connection_frame, the connection's dispatcher, send) while another thread holds the client registry lock for 300 ms; each key must reach its PTY in under 150 ms.
  • server::tests::rds_input_lane::a_key_waits_for_neither_the_session_nor_the_workspace_registry_lock: the same with Mux::state and the workspace registry (they stay off the key on the new path; #31940).
  • server::tests::rds_input_lane::after_a_respawn_the_next_key_reaches_the_new_terminal_on_the_input_lane: the published kept terminal follows the route rule of #31940 (a respawn under the same id sends the next key to the new shell, never to the replaced one).
  • published::tests (3): a change publishes on drop and a read publishes nothing; a reader does not wait for a writer that holds the value; a poisoned value gives its projection in the error.
  • The two remote-relay poison tests (a_poisoned_registry_lock_trusts_no_client_as_local, a_poisoned_registry_trusts_no_unix_client_and_still_removes_a_record) poison through the new type and run on the default path.

The earlier census commit 2472251831862 (all nine shared locks, one test) is in the branch history; its hosted run https://github.com/manaflow-ai/cmux/actions/runs/41648095321 measured 300 ms waits for 7 of 9 locks (client registry, link peers, host proxy authors, surface admission, creating input, client sizing, activity) and 0.1 ms for the two controls. The other six locks are the next S1 steps (DESIGN.md section 3).

Observed / Not run

  • Observed: the hosted Linux runs named above (red at 7dc4598d221, green at the head), and the fleet compile check c6563617ceaaca223723ffc2 (CmuxNext, app and cmux-tui passed).
  • Not run: the new tests on macOS (blocked by the acpmux base red above); the full cmux-tui suite with the switch on; the real-app A/B (typing-load-l6, base vs switch on, interleaved on one host). That A/B decides the default flip (stage S1.7). Perf claim: none in this PR.
  • Base reds in the same hosted runs that this PR does not touch: the lint (linux) job fails on acpmux clippy unused import notify::Watcher at crates/acpmux/src/login_env/watch.rs:15 (from #33406; #33690 fixes it); cmux-tui-core itself passed clippy in that job with --keep-going (Checking cmux-tui-core then Finished), and its panic ratchet counts are the same as the base run's (arithmetic_side_effects 1002, cast_possible_truncation 100, both runs). The macOS acpmux web_image_attachments test aborts while it lists tests on the base too.

Changelog

none

Proof

No capture: daemon-only change behind a default-off switch, no UI.