#23113 cmux-tui: frame producers and browser workers share one executor, not a thread per surface (perf100)

perf100 tui lane, root cause 2: a thread per entity that waits for that entity's work. Stacked on #22973 (review that first; this PR's own commits start at "render frame producers").

Every terminal had a surface-<id>-frames thread parked on a channel, and every browser surface a browser-surface-<id>-worker thread parked for good. Lawrence's Chief daemon had 335 + 353 of them.

Change

• serial_lanes.rs: one shared executor for per-entity serial work. A lane's run function runs on a shared worker when kicked or at its deadline. It never runs on two threads at once, and a kick during a run makes it run once more. Workers grow only with work in flight: a run that blocks gets a fresh worker for the rest. Extra workers exit after 10 s idle, so an idle daemon keeps one worker, blocked with no timeout. A closed lane drops its state at once. Each lane keeps one armed deadline per instant.
• The frame producer becomes a lane. The 8 ms cadence is now a deadline, and no request can be lost: a request after the lane takes the owed flag kicks it again.
• The browser worker becomes a lane. BrowserCommandTx kicks on send, and the last sender kicks on drop so the lane reads on to Disconnected. Lifecycle deadlines become lane deadlines. Batching, ordering and done_tx stay as they were.
• thread_budget.rs budget: 2.25 threads per terminal (was 3.25 in #22973; 4 on base).

Evidence (rbx, Linux, release build, scripts/cmux-next/bench_tui_service.py, base 78aaa620aa5e vs this head)

Metric	Base	After	Ratio
S3 daemon threads, 1000 terminals	4025	2027	1.99x
S3 daemon threads per terminal	4.02	2.02	1.99x
S3 daemon RSS, 1000 terminals	236 MB	208 MB	1.13x
S0 daemon threads, 10 attached terminals	119	101	1.18x
S0 idle daemon context switches, 60 s	11	12	same (floor)
thread_budget per terminal	4.00	2.00	2x

The rest of the daemon's threads per terminal are the host stream reader and its demultiplexer (2), and 5 per attach connection. Those are the next root causes.

Observed / Not run

• Ran on rbx: clippy -D warnings (cmux-tui-core, cmux-tui); cargo nextest -p cmux-tui-core (2771 pass; 5 fail, the same 5 fail on base); thread_budget, idle_wakeups, terminal_resources (all pass).
• macOS: hosted focused runs 39546053711 and 39548051079 were dispatched. The Linux lint job failed on a base clippy red in acpmux (steer_now_tests.rs). The macOS jobs were still queued after 58 min. macOS is UNVERIFIED.
• Review: subagent findings and how each was fixed are in #22973's comment.

Changelog

none

🤖 Generated with Claude Code

---

^{Need help on this PR? Tag @codesmith-bot with what you need.}

---

Summary by cubic

Moves browser-surface workers off dedicated threads onto a shared executor, and terminal-host reaping onto one shared thread, cutting daemon threads per terminal from about 4 to about 2 with rendering, batching, and ordering unchanged. The new lane and browser-worker code adds no unwrap or expect: a poisoned lock recovers its state.

Changes

• New serial_lanes.rs: a shared executor runs one entity's work at kick or deadline, never on two threads at once; a kick during a run makes it run once more, and idle workers exit after 10 s.
• Browser workers become lanes, keeping lifecycle deadlines, command batching, and done_tx signaling. Only the last browser command sender kicks on drop, so the lane reads on to Disconnected.
• One terminal-host-reaper thread waits on kqueue (Apple) or a pidfd in epoll (Linux) for all hosts; other platforms fall back to the old per-host wait4 thread. After an early exit event the reaper re-arms, since XNU posts NOTE_EXIT before the child is a zombie: the kernel refuses an exiting process (ESRCH), and only then does it wait.
• Added thread_budget.rs (2.25 threads per terminal budget, host reaping verified) and bench_tui_service.py, which measures daemon threads, RSS, CPU, and context switches across S0/S1/S3 benchmarks (including CPU of reaped hosts).

Validation

• Linux rbx run passed: clippy with -D warnings, cargo nextest (2771 passed; 5 failed, the same on base), thread_budget, idle_wakeups, and terminal_resources.
• macOS unverified: the macOS jobs were still queued when the PR was opened; the Linux lint job failed on a pre-existing clippy error in acpmux at the base revision.