thread rail: frame-cost bench for long chats

Merged
TE
12d ago
+353-0

This PR adds a bench for the thread rail's cost per frame on long chats. It adds no product code: one file, webviews/bench/thread-minimap-frame-cost.bench.ts. It was split out of #25152 to keep that PR under 800 lines. #25152 cites these numbers.

Leo (2026-10-25): "the mini map VS Code code thing ... I feel like we should have that. Although I don't know if that actually, like, causes potential lag."

What it measures

  • Harness: the real agent pane talks to a stand-in acpmux holding a chat of N turns (2N rows; the default is 1,000 turns, so 2,000 rows), in headless Chromium at 1200x800.
  • Arms: each run sets thread-minimap-layout in cmux.experiments to off, compact (the wave rail, the default) or full (the #22784 minimap).
  • Scenarios, 4 s each:
    • idle;
    • scroll: a wheel step of -120 every 16 ms;
    • stream: a reply streaming 30 chunks/s, with a new prompt and reply every second.
  • Per frame: a Chrome trace of the renderer main thread, split at painted-frame stamps (rAF console.timeStamp). For each frame it records:
    • busy time (top-level tasks);
    • scripting time and rendering time (style, layout, paint, layerize, hit test);
    • frame interval.
  • Output: p50, p95 and mean of each, the frame count, long tasks over 50 ms, and each arm's extra cost over off. Each figure is the median across rounds.

How to run

cd webviews
# PROD_DIR is required: a production build of the pane's dev entry (vite's build() with
# vite.config.acpmux-pane.mjs; the directory holding its index.html), served as static files, so
# React runs as it ships. Without it the bench exits 2 with a message.
PROD_DIR=DIR bun bench/thread-minimap-frame-cost.bench.ts [rounds=3] [turns=1000]

Options:

  • ARMS=off,compact runs only those arms.
  • PROFILE=1 also writes a V8 CPU profile per run.
  • TRACE_CATS=... adds trace categories.
  • EXTRA_CSS=... adds a stylesheet, to find where a cost comes from.
  • KEEP_TRACES=1 keeps the traces and profiles in $TMPDIR/minimap-bench-*; otherwise they are removed.

The stand-in acpmux, the static server, the browser and the trace directory are released in a finally, in reverse order and each on its own. A failed launch, navigation, trace or analysis, or a failing close, still releases the rest. Checked by hand: a missing browser and a navigation timeout both exit 1, leaving no trace directory and no browser.

Numbers (huge-blue, 3 rounds, 1,000 turns, production build)

Extra main-thread cost per frame over the rail being off, p95 (ms):

idlescrollstream
compact rail, feat-cmux-next+0.01+35.5+43.6
compact rail with #25152+0.01+3.0+5.8
full-height minimap (#22784), feat-cmux-next+0.02+57+112

Rail off alone runs about 18ā€“20 ms (scroll) and 20ā€“22 ms (stream) at p95. Frames painted in 4 s:

  • compact on base: 145 while scrolling and 95 while streaming;
  • with #25152: 242 and 237;
  • full minimap: 120 and 42.

Risk

None to the product: the file is not in any build, bundle or test glob, and CI does not run it.

Agent session: huge-blue

šŸ¤– Generated with Claude Code