cmux vnc: a Rust VNC viewer for Cloud machines, Macs, Windows and Linux in a pane

Merged
AU
8d ago
+6396-0

Fixes #25580

Summary

A Rust VNC (RFB) client, and cmux vnc, which shows and drives a remote screen inside a cmux terminal pane.

  • cmux-tui/crates/cmux-vnc: the RFB client. RFB 3.3, 3.7 and 3.8 (Apple's 3.889 included). Security: none, VNC password, Apple DH (type 30, macOS Screen Sharing with a Mac user name), VeNCrypt without TLS (None, VncAuth, Plain). Decoders: Raw, CopyRect, RRE, Hextile, Zlib, ZRLE, lossless Tight (no JPEG is requested). Pseudo-encodings: Cursor, DesktopSize, ExtendedDesktopSize, LastRect, DesktopName. Input: X11 keysyms, pointer and wheel, clipboard, SetDesktopSize. Transports: TCP or a proxy command's stdio. A reconnect backoff that never retries an authentication failure. Every server-supplied size and length is bounded.
  • cmux vnc <address> (cmux-tui CLI): draws the screen with the Kitty graphics protocol, scaled to fit the pane and centered, and sends keys and mouse back.
    • cmux vnc machine:<id> opens a Cloud machine's desktop (TigerVNC on 5901) through cmux cloud port-forward, at every connect and reconnect.
    • host, host:N (display), host::port, [v6]:port, vnc://, or --proxy-command 'ssh -W localhost:5901 box'.
    • Resize: --resize auto (default) asks the server to match the pane when it supports SetDesktopSize (TigerVNC); otherwise the picture scales (macOS, most Windows servers).
    • Reconnect: a dropped connection retries with backoff (250 ms doubling to 10 s); the delay resets only after a session lives 10 s; an authentication failure, or three protocol errors in a row, stops and says why.
    • Keyboard: with the Kitty keyboard protocol keys go down and up as held, modifiers included; other terminals get press and release inside the named modifiers. Ctrl-] then q quit, r reconnect, v view only, f redraw, Ctrl-] sends Ctrl-].
    • Mouse: SGR-Pixels where the terminal reports it (Ghostty, kitty, WezTerm), else cell centers. Paste types the text and sets the remote clipboard; --remote-clipboard also copies the remote clipboard back over OSC 52 (off by default, so a server cannot set your clipboard).

Defaults I picked

  • The viewer lives in the cmux CLI and draws with Kitty graphics, so it works in any cmux terminal pane (NEXT, classic, cmux-tui, and inside a Cloud machine's own session) with no app change. The Swift remote_view pane and CmuxVNC are untouched.
  • Shared session (other viewers stay connected), local cursor drawing, ZRLE first, 30 fps cap, the picture never scaled up.
  • Credentials: --username / $CMUX_VNC_USERNAME, --password-env (default CMUX_VNC_PASSWORD) or --password-file; if the server asks and none is set, the CLI asks in the pane (password not echoed) before the viewer starts.
  • Server text (desktop name, refusal reasons) is printed with control characters replaced, so a server cannot send terminal escapes. Input goes through a bounded writer thread: a stalled network never freezes the viewer.
  • RealVNC's own RA2 security and VeNCrypt TLS subtypes are refused with a message that says what to enable on the server.

Testing

The reviewed head is c71ab38fd20fcdebec114866457b99cb8ea57698, including merged base bcec3ce42baf5f9458d232e1441d828b27f2641b. An independent correctness review checked both merge parents and the complete 35-file feature diff: VNC source/tests/highlight and the four shared Cargo/CLI integration files are unchanged by this merge, and base repairs #29075 (Windows), #28888 (OptChat) and #29154 (release/notary) are retained. On this exact head, rbx passed all 44 protocol/session tests, 14 CLI tests, formatting, scoped all-target Clippy with warnings denied, and cargo check --locked -p cmux-vnc -p cmux-tui --lib --bins --target x86_64-pc-windows-gnu (exit 0; existing Windows warnings remain). The remote checkout explicitly asserted the tested SHA; the launcher's local-worktree footer still names the previous local head. The authoritative fleet Release compile passed on this exact SHA; the public compile receipt names job 18f17df7d8401fe9bbd7a75f and state pass. The current iOS Rust failure is the same browser-capability test failure as the base job, confirmed with test_reds.py classify. Pending CI is not claimed as passed.

  • Red/green regression: committed the failing tests in 9ea6d56f4ca48c215689f80225498cda633e7ef9, then repaired them in 4a7c612915b. On rbx, cargo test --locked -p cmux-vnc vnc_ --no-fail-fast failed both zero-dimension tests (including the decoder panic), and cargo test --locked -p cmux-tui vnc_kitty_releases_held_keys_while_awaiting_a_viewer_command failed the remote key-release assertion. The same commands passed after repair. Zero dimensions now end the session with a protocol error, and held keys are released while a Ctrl-] command is pending.

  • The original panic-ratchet CI failure is repaired with checked access and iterators, with no baseline change. The full workspace ratchet passed at d1b44c5826474b9529433ce3b20b98e854dfa23a (76 crates, 25 at deny). At the prior merged head 3bbb02fa11764eb9376eeb54b12348c5507b5ea6, it checked 79 crates (26 at deny) and reported inherited findings in cmux-automations, cmux-rd-link, cmux-tui-core, cmux-chats-mirror, and cli/automation.rs:101. All failing files, the ratchet, and baseline are identical to base 9962e57684c30d6e82fccc866e4e1a5630ecdf88. Neither VNC path has denied findings. Only the DES S-box and MD5 fixed-size algorithms have function-scoped exceptions, independently reviewed against their bounds.

  • Large-image regression: 071e92c3a7b49c6dd3d57281c7dfa151ee6ff1c4 adds cargo test --locked -p cmux-tui --lib vnc_scaling_large_white_image_does_not_overflow. It failed on rbx with attempt to add with overflow for a 4112×4112 white screen averaged into one pixel. It passes after widening the color sums to u64 and validating scaling inputs in d1b44c5826474b9529433ce3b20b98e854dfa23a.

  • python3 scripts/whats-new/test_digest.py: 5 passed after adding the highlight's docs link, fixing the release-note CI failure.

  • The implementation pushes passed python3 scripts/cmux-next/prepush-gate.py and the normal push hooks with no new violations. Scoped verify-local.py previously passed 9 checks (including 17 project tests) at merged base 8fce89d2a85e10161fac2fab1b3dd408d5997169; localization parity was red on base debt. The later scripts/localize-changes --base origin/feat-cmux-next run at base 397ab560253ef9dcd2cae14339a23133196c1452 stopped on 47,003 existing catalog parity findings. This PR changes no catalogs or localization inputs. Rust CLI translation coverage remains unverified.

  • cargo test --locked -p cmux-vnc on the Rust builder (rbx): 44 passed. 35 unit tests (DES and MD5 reference vectors, Apple DH checked by a server-side decrypt, each decoder and subencoding, pixel formats, endpoint parsing, backoff) and 9 end-to-end sessions (two check that a quit sends queued key releases first, over TCP and through a slow proxy) against in-process servers that answer the way each guest OS's server does:

    • macOS Screen Sharing: RFB 003.889, security [30, 33, 36, 35], Apple DH, ZRLE frame, Command-Q as Super_L + q; without a user name the client says it needs one and does not retry.
    • Windows: VNC password accepted and refused (reason surfaced, no retry), Tight fill, clipboard both ways; an RFB 3.3 server that picks the security type, LastRect, Bell, DesktopSize.
    • Linux TigerVNC: render, keys, pointer, SetDesktopSize accepted and refused, the server going away, reconnect; VeNCrypt Plain.
    • Hostile servers: a 60000x60000 framebuffer, a rectangle outside the framebuffer, a non-RFB banner, a zlib bomb.

  • cargo test --locked -p cmux-tui --lib vnc_ on rbx: 14 passed (arguments, the port-forward answer, layout and pointer mapping, scaling, Kitty chunking and image swap, both keyboard modes, a Kitty release after Shift changed, Ctrl-5 vs Ctrl-], Ctrl-] commands, escape stripping).

  • cargo clippy --locked -p cmux-vnc -p cmux-tui --all-targets -- -D warnings and cargo fmt --check -p cmux-vnc -p cmux-tui on rbx: clean. On the earlier repair head d1b44c5826474b9529433ce3b20b98e854dfa23a, Windows cross-compilation passed for both libraries and the CLI binary (cargo check --locked --target x86_64-pc-windows-gnu --lib --bins -p cmux-vnc -p cmux-tui) and all VNC tests (cargo check --tests --locked --target x86_64-pc-windows-gnu -p cmux-vnc). The broader Windows --tests check stops in unchanged base test app_refusal_note.rs:8, which imports std::os::unix::net::UnixListener.

  • Earlier live probe, TigerVNC Xvnc 1.13.1 on the Rust builder, driven by cargo run -p cmux-vnc --example probe: connected (RFB 3.8, none), first frame, a click landed at (100, 100), typed text arrived in an xterm, SetDesktopSize to 1024x700 took effect (xdpyinfo), reconnected after Xvnc was killed and restarted (backoff 500 ms, 1 s), VNC password accepted and a wrong one refused with the server's reason.

  • The existing python3 cmux-tui/scripts/check-resource-api-boundary.py failed on the viewer diagnostic “invalid VNC image slot”; this is a local Kitty image array index. Renaming the diagnostic to “invalid VNC image index” passes the same checker (351 public files scanned at 3bbb02fa11764eb9376eeb54b12348c5507b5ea6), with no scanner exception or runtime change.

  • A prior rbx rerun exposed a test-harness race in close_unblocks_the_reader: immediate close could legitimately discard queued setup before the fake server read it. The same cargo test --locked -p cmux-vnc command passed after adding a server-setup readiness signal, preserving the reader-close assertion.

  • python3 tests/test_ci_signing_owned_mac.py SigningHooksInStepWithGlaedaTests: 1 passed at 3bbb02fa11764eb9376eeb54b12348c5507b5ea6, after merging base 9962e57684c30d6e82fccc866e4e1a5630ecdf88. This is the exact signing-hook test that previously blocked autoland.

Observed / Not run

  • Observed: everything listed under Testing.
  • Not run: a live macOS Screen Sharing session and a live Windows server (both need machine credentials); they are covered by the protocol-faithful fake servers above. The Kitty-graphics drawing was checked by unit tests of the escape output, not by a screenshot of a pane; no visual pane capture was produced.

No capture: this changes the terminal CLI and protocol client; no native app UI changed.

Changelog

  • Added: cmux vnc shows and drives a VNC screen (Cloud machine, Mac, Windows or Linux) in a terminal pane.