cmux-next agent pane: compare the transcript against a Codex capture

Merged
TE
1mo ago
+419-47

Part of the cmux-next parity work listed in #16174 (comment). Stacked on #16578 (mock mode), which GitHub retargets to feat-cmux-next when it merges.

Problem

The agent pane polish wave ports codex-atlas-clone's transcript components and needs a fast, repeatable score of how far the pane is from Codex. Fleet builds take too long per iteration, and there was no way to render the pane outside the app with a real turn and the app's theme.

Change

  • bun run compare:agent-pane [scenario] (webviews/scripts/agent-pane/compare.mjs) starts the pane's Vite dev config, opens it in headless Chromium with the host bridge stubbed so ready answers mock, sends the scenario's prompt, and screenshots the pane at the size of the reference's content area at 2x.
  • The turn goes through the production path: MockAcpmuxSocket can replay a recorded turn (window.cmuxAcpmuxMockScript, mock mode only). Each step carries its offset from the turn start, so durations read as recorded. A replayed session starts empty, without the mock greeting.
  • scripts/agent-pane/theme.mjs derives the bridge theme from the default Ghostty theme the way ThemeTokens.derive and AgentPaneTheme.values do, so the harness colors the pane as the app does.
  • The transcript is scrolled until an anchor line sits where the reference has it, then the area above the composer is scored with pixelmatch (threshold 0.1), the same metric as codex-atlas-clone's compare-region.mjs. It writes ref, actual, diff and side-by-side PNGs.
  • First scenario: specimen-complete, the markdown specimen turn (commentary, a failed tool call, the answer) from codex-atlas-clone's recorded rollout.
  • References stay in the private codex-atlas-clone checkout (--atlas or CMUX_AGENT_PANE_ATLAS). Nothing from it is committed here except the turn's text.

Dev dependencies: playwright 1.62.1 (same as web/), pixelmatch, pngjs.

Numbers

specimen-complete: 2.838% mismatched at threshold 0.1 (the atlas metric), 96.445% at 0.02 (shows the background color cast). This is the baseline the transcript port PRs report against.

Verification

  • bun test src/agent-session/acpmux: 90 pass, including a new mock test that a recorded turn replays with its timestamps and no greeting.
  • bun run lint:ci, bun run typecheck, build-agent-pane-web.sh --check clean.
  • Ran the harness on Linux; the side-by-side shows the pane aligned on the specimen heading.

No UI change in the app: the only runtime change is a mock-mode option.

🤖 Generated with Claude Code