cmux NEXT perf: one harness times splits, the side pane, the sidebar, a new workspace at the top and switches (app-perf)
Leo-ask: ask-995a5ef560
Leo (2026-10-21): "the fellas and i we were talking about systemically why are things slow how can they get faster / e.g. splitting, e.g. cmd shift d, e.g. adding stuff to sidebar, all that... / personally, i was thinking how some companies/apps have rendering systems, and about games -- well why can they do things faster? because they have animations and logic for recalculating bundled in / i think we're frequently just recalculating too often, too unnecessarily / there's more with virtualizing things... i imagine we're not really virtualizing too much of our sidebar. right now adding a workspace at the top ... it pushes all the stuff down"
This is step 1 of that program: one harness that measures those gestures, so each fix PR can show a before and after number from the same harness.
What it measures
scripts/cmux-next/app-perf.py launches a tagged cmux-next app (scratch config), holds 10 and then 50 terminal workspaces, and times each gesture from the press until the window rests:
| scenario | gesture |
|---|---|
split-right | Cmd-D (real key path, debug.key), then the close (untimed) |
split-down | Cmd-Shift-D, then the close |
right-sidebar-open / -close | Cmd-Opt-B, the side pane (read from the main pane's width) |
sidebar-hide / -show | Cmd-B |
new-top | New Workspace at Top above the existing 10 or 50 |
switch | goToWorkspace between the first and the last workspace |
Per scenario: frame interval p50/p95/p99/max, missed vsyncs, frames dropped at 120 Hz, main-thread busy ms per frame (p50/p95/max and the gesture's total), NSView.layout() calls and window layout passes per frame, run-loop gaps over 8.3 ms, hangs, the busiest layout classes, the sidebar and layout work counters (debug.sidebar_perf, debug.layout_counters), and the app's own press-to-commit ms for new-top. It prints a Markdown table and a REPORT-JSON line; the JSON keeps every frame.
A run passes only as a full set (RESULT PASS): every requested workspace count fills, and every round of every scenario does what it names (a split made a pane; each toggle round has both halves and each press changed the main pane's width; new-top's single new workspace is the sidebar's first workspace row, read from debug.sidebar_rows; a switch landed). Otherwise RESULT FAIL lists the reasons in the output and in failures. Slow numbers are data, not failures.
Dropped at 120 Hz: on a 120 Hz display it counts the vsyncs each interval lost; on a 60 Hz console it estimates per frame the larger of the 120 Hz frames the interval lost and the 120 Hz frames the main thread's work overran (work over 8.33 ms misses a 120 Hz vsync). The table header names the display rate it ran at.
App side (DEBUG builds only)
debug.frameskeeps, per frame, the main run loop's awake time (twoCFRunLoopObservers, wake and before-wait) and the deltas of two newLayoutPassGuardcounters.raw: truereturns the per-frame arrays. Stop condition and cleanup: the observers exist only betweenstartandstop;stop, or the probe's existing 600 s expiry, invalidates and removes them. No timer or display link is added (the probe's existingFrameClientticks).LayoutPassGuard.viewLayoutCalls/windowLayoutPasses: plain integer adds inside the existing-[NSView layout]/-[NSWindow layoutIfNeeded]swizzle, which already updates a dictionary per call. Compiled under#if DEBUG; a release build always reads 0. A window pass counts only when it laid out a view.
Run it (fleet, alone on an owned mini's console)
cmux-ci run --class exclusive --measurement --label measure --script scripts/measure/split-spring.sh \
--ref <full sha> --key app-perf-<sha12> --arg=--suite=app-perf --artifact artifacts/split-spring/report.json
scripts/measure/split-spring.sh is the allowlisted entry; split-spring-terminal.py --suite=app-perf hands the run to app-perf.py, the way --suite=pane-perf does. Options: --arg=--workspaces=10,50, --arg=--rounds=5, --arg=--scenarios=split-right,new-top, --arg=--input=action.
No capture: no UI change (a debug probe and a script).
Sample run at this head
Script at ccaa3a0fd340 on the fleet build of bd7d426075d1 (job 07e085411c1223a738507cf1, tag apperf; the Swift is identical at both heads, only app-perf.py changed). Run on capture mini cmux-mac-mini-cua-3 (capture-host lease, --open):
python3 -I app-perf.py --tag apperf --app 'app/cmux DEV apperf.app' --open --sha bd7d426075d10341c0bed6205156adc622c2ecd5 --workspaces 10,50 --rounds 3 --markdown table.md
It printed closing 52 workspaces an earlier run left, then 55 of 55 rows ok and RESULT PASS. The run's files and the mini's tags/apperf state were removed afterwards.
The sample I posted earlier (at c6fda1223b89) is superseded and its numbers are wrong. Its new-top closes were refused (closeWorkspace needs confirm from a script), so later rounds ran with extra workspaces. The restore check added since then caught that.
Output table
app-perf at bd7d426075d1 on cmux-austin-mini-1 (macOS 27.0.1), display 60 Hz, 3 rounds, input key. Frame and main-thread columns pool every round's frames; totals are medians per gesture.
| ws | scenario | ok | frame p50 | p95 | p99 | max ms | missed | drop@120 | main ms/frame p50 | p95 | max | main ms total | view layouts/frame p95 | max | view layouts total | window passes/frame max | window passes total | gaps>8.3ms | hangs | app ms |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 10 | split-right | 3/3 | 16.7 | 16.7 | 33.3 | 33.3 | 2 | 9 | 0.25 | 11.63 | 24.50 | 167.3 | 14 | 64 | 271 | 5 | 54 | 4 | 0 | - |
| 10 | split-down | 3/3 | 16.7 | 16.7 | 33.3 | 33.3 | 2 | 10 | 0.30 | 12.46 | 21.65 | 158.9 | 14 | 64 | 270 | 5 | 54 | 5 | 0 | - |
| 10 | right-sidebar-open | 3/3 | 16.7 | 16.7 | 33.3 | 78.9 | 1 | 4 | 0.17 | 3.38 | 67.46 | 62.7 | 1 | 82 | 55 | 8 | 10 | 2 | 1 | - |
| 10 | right-sidebar-close | 3/3 | 16.7 | 16.7 | 33.5 | 39.4 | 2 | 6 | 0.16 | 5.05 | 28.10 | 80.4 | 1 | 45 | 54 | 8 | 10 | 2 | 0 | - |
| 10 | sidebar-hide | 3/3 | 16.7 | 16.7 | 33.3 | 33.3 | 2 | 12 | 0.21 | 14.68 | 24.52 | 145.3 | 55 | 56 | 337 | 9 | 55 | 7 | 0 | - |
| 10 | sidebar-show | 3/3 | 16.7 | 16.7 | 33.3 | 33.3 | 1 | 13 | 0.22 | 14.35 | 28.73 | 169.9 | 56 | 62 | 464 | 9 | 75 | 9 | 0 | - |
| 10 | new-top | 3/3 | 16.7 | 29.0 | 44.3 | 48.6 | 5 | 17 | 0.14 | 16.59 | 70.38 | 170.9 | 9 | 80 | 142 | 5 | 14 | 7 | 0 | 22.7 |
| 10 | switch | 6/6 | 16.7 | 16.7 | 33.3 | 51.0 | 1 | 4 | 0.15 | 2.32 | 40.99 | 56.1 | 0 | 18 | 18 | 3 | 3 | 2 | 0 | - |
| 50 | split-right | 3/3 | 16.7 | 16.7 | 33.3 | 33.3 | 2 | 11 | 0.24 | 13.28 | 30.41 | 186.8 | 14 | 70 | 271 | 6 | 54 | 4 | 0 | - |
| 50 | split-down | 3/3 | 16.7 | 16.7 | 33.3 | 33.3 | 2 | 12 | 0.25 | 12.60 | 25.28 | 189.1 | 14 | 64 | 271 | 5 | 54 | 5 | 0 | - |
| 50 | right-sidebar-open | 3/3 | 16.7 | 16.7 | 33.9 | 39.4 | 1 | 4 | 0.17 | 5.57 | 32.39 | 72.9 | 1 | 46 | 55 | 9 | 11 | 2 | 0 | - |
| 50 | right-sidebar-close | 3/3 | 16.7 | 16.7 | 33.3 | 41.1 | 2 | 6 | 0.17 | 4.16 | 29.75 | 84.9 | 1 | 46 | 55 | 9 | 11 | 2 | 0 | - |
| 50 | sidebar-hide | 3/3 | 16.7 | 16.7 | 33.3 | 33.3 | 1 | 10 | 0.20 | 14.42 | 24.44 | 139.7 | 56 | 57 | 342 | 10 | 61 | 8 | 0 | - |
| 50 | sidebar-show | 3/3 | 16.7 | 16.7 | 33.3 | 50 | 2 | 13 | 0.29 | 13.99 | 31.50 | 159.4 | 57 | 63 | 472 | 10 | 83 | 9 | 0 | - |
| 50 | new-top | 3/3 | 16.7 | 33.3 | 50.4 | 59.8 | 4 | 23 | 0.15 | 29.16 | 61.55 | 194.0 | 11 | 85 | 144 | 6 | 15 | 8 | 0 | 27.2 |
| 50 | switch | 6/6 | 16.7 | 16.7 | 33.7 | 48.3 | 1 | 4 | 0.15 | 5.39 | 46.11 | 101.1 | 4 | 15 | 47 | 3 | 17 | 2 | 0 | - |
Busiest layout classes per gesture (median NSView.layout calls):
- 10 ws split-right: _NSCoreHostingView 38, NSScrollPocket 35, NSGlassEffectContainerView 32, _NSCoreHostingView 32, NSThemeFrame 19, NSView 11
- 10 ws split-down: _NSCoreHostingView 38, NSScrollPocket 35, NSGlassEffectContainerView 32, _NSCoreHostingView 32, NSThemeFrame 18, NSView 11
- 10 ws right-sidebar-open: NSScrollPocket 8, NSView 3, NSThemeFrame 2, _NSCoreHostingView 2, NSGlassEffectContainerView 2, HomeSidebarSplitView 2
- 10 ws right-sidebar-close: NSScrollPocket 8, NSView 3, NSThemeFrame 2, _NSCoreHostingView 2, NSGlassEffectContainerView 2, HomeSidebarSplitView 2
- 10 ws sidebar-hide: NSScrollPocket 54, NSView 24, TitlebarBandButton 19, NSThemeFrame 13, _NSCoreHostingView 12, NSGlassEffectContainerView 12
- 10 ws sidebar-show: NSScrollPocket 73, NSView 32, TitlebarBandButton 25, NSThemeFrame 17, _NSCoreHostingView 16, NSGlassEffectContainerView 16
- 10 ws new-top: NSThemeFrame 10, NSView 8, _NSCoreHostingView 6, WorkspaceStatusView 6, HoverRevealProbe 6, TabStripView 6
- 10 ws switch: NSThemeFrame 2, _NSCoreHostingView 2, NSGlassEffectContainerView 2, NSNextStepFrame 2, DividerMouseCatcherView 2, _NSCoreHostingView 2
- 50 ws split-right: _NSCoreHostingView 38, NSScrollPocket 35, NSGlassEffectContainerView 32, _NSCoreHostingView 32, NSThemeFrame 19, NSView 11
- 50 ws split-down: _NSCoreHostingView 38, NSScrollPocket 35, NSGlassEffectContainerView 32, _NSCoreHostingView 32, NSThemeFrame 19, NSView 11
- 50 ws right-sidebar-open: NSScrollPocket 9, NSView 3, NSThemeFrame 2, _NSCoreHostingView 2, NSGlassEffectContainerView 2, _NSCoreHostingView 2
- 50 ws right-sidebar-close: NSScrollPocket 9, NSView 3, NSThemeFrame 2, _NSCoreHostingView 2, NSGlassEffectContainerView 2, _NSCoreHostingView 2
- 50 ws sidebar-hide: NSScrollPocket 60, NSView 24, TitlebarBandButton 19, NSThemeFrame 13, _NSCoreHostingView 12, NSGlassEffectContainerView 12
- 50 ws sidebar-show: NSScrollPocket 81, NSView 32, TitlebarBandButton 25, NSThemeFrame 17, _NSCoreHostingView 16, NSGlassEffectContainerView 16
- 50 ws new-top: NSThemeFrame 11, NSView 8, _NSCoreHostingView 6, WorkspaceStatusView 6, HoverRevealProbe 6, TabStripView 6
- 50 ws switch: NSThemeFrame 16, WorkspaceRowView 10, UnreadBadgeView 2, _NSCoreHostingView 2, NSGlassEffectContainerView 2, DividerMouseCatcherView 2
Agent session: cosmic-gray
🤖 Generated with Claude Code