PR #29080 - manaflow-ai/cmux

Summary

Tool-eval hill climb, rounds 14 to 16, for the cmux MCP server (cmux-tui/crates/cmux-tui/src/cli/mcp) and tests/tool-evals.

open tool (new): opens a file, a diff or a web page where the person asked, in one call. where is right, left, below, above, here, sticky (a docked column), pane or workspace. Before this tool, every open took 2 to 4 steps (about 20k tokens each), and no model found how to make a sticky column.
- A page targets the pane's shown tab, because tab new-browser takes a tab target.
- A relative path is joined to the chat terminal's folder, because the app opens only absolute paths.
- sticky skips a column that has a dock role (the agent chat's) and skips the chat's own column.
- A refused open closes the pane that it split.
tab_move: takes just a destination pane or just a workspace. A workspace with no pane names pane_create.
Code-mode sheet: names open, tab_move, workspace_create, terminal_input_keys, and how to make an existing column sticky.
tool-evals:
- 20 new open-placement cases (cases/open.ndjson).
- A stand-in app that runs the app's open actions and refuses what the real app refuses.
- An app-socket relay, so the remote MCP server can reach the stand-in app. Tailscale SSH forwards the socket as root 0600, so every remote run before this one had no app.
- Median tokens and seconds per case in the summary.

Results (open cases, 20 cases x 2 samples, median per task)

Model, version	Pass	Tokens	Seconds	Steps
Sonnet 5.5, old sheet	10/40	100k	19.5	6
Sonnet 5.5, `open` tool	36/40	56k	27.2	3
Opus 5.5, `open` tool	39/40	75k	15.4	4

Review

A correctness subagent reviewed the open tool. Its findings and their resolution are in a PR comment.

Changelog

none

Observed / Not run

Observed: on rbx, cargo test -p cmux-tui cli::mcp and clippy -D warnings on each pushed head. Opus and Sonnet eval runs on the open cases (the numbers above).
Not run: the eval runs for GPT 6 Astra, GPT 6.1 Sol and the OpenRouter models on the open cases. They follow this PR.

Follow-ups

Eval harness socket mode (owner: lawrencecchen, the tool-evals harness author of this PR). tests/tool-evals/session.mjs line 117 runs sudo -n chmod 666 on the forwarded stand-in app socket (<app>.fwd). While a run is active, any user on the eval host can reach the stand-in app. Fix: give the socket to the eval user only (owner-only mode 600, or mode 660 with a dedicated group). Do not make it world-writable. Found by the refresh review at 941db130d04 (https://github.com/manaflow-ai/cmux/pull/29080#issuecomment-6546360645).

^{Need help on this PR? Tag @codesmith-bot with what you need.}

Summary by cubic

Adds an open tool to the cmux MCP server that places a file, diff, or web page where the user asked in one call, replacing the 2–4 step workaround that never produced a sticky column.

Changes

open targets right, left, below, above, here, sticky (docked column), a named pane, or a workspace; several opens to the right share one column, and a refused open closes the split it made.
Pages open beside a pane's shown tab, relative paths resolve against the chat terminal's folder, and the chat's own column is never reused as sticky.
tab_move now needs only a destination pane or workspace; a workspace with no pane gets a clear refusal naming pane_create.
The code-mode sheet names open, tab_move, workspace_create, terminal_input_keys, and how to dock an existing column.

Tool-evals

Adds 20 open-placement cases plus a stand-in app that runs the app's open actions and refuses what the real app refuses.
Adds a socket relay so remote MCP runs can reach the stand-in app (Tailscale SSH forwarded the app socket as root 0600 before).
Stops a remote sample's daemon on the remote host too; a stop through a closed forward had left 1,000+ daemons running on rbx.
Reports median tokens, seconds, and steps per case; the screenshot-honesty case now accepts "isn't/aren't available, connected, supported" phrasing.