ci(webviews): browser test files run in a pool of their own, a quarter of the jobs wide
Red fix for react-apps-check: its shards' browser test files fail under load and pass alone.
What was wrong
The shard runner started its plain (non dev-server) files 32 at a time, longest first. The longest plain files are the Playwright ones (each launches Chromium and WebKit), so on a 32 vCPU shard nearly every browser file started together: 29 of the first 32 files on #27329's shard 1 (job 123385326827). Eight timed out at 120 s, frame-timing plays failed (agent-pane-wheel-glide: "frame time 71 vs <60", "painted frames: 1/2, needs >=3"), chat-inspector-placement shifted 111 px, and every one of them passed when react-apps-base-check replayed it one at a time (job 123394574731). Those first-run failures then used up the shared retry budget, so a later failed file's retry got 3 s ("RETRY test/agent-pane-wheel-glide.test.ts (remaining budget 3s, hard limit 3s)" then TIMEOUT). The same wave shows on base pushes: on shard 2 of two base push runs, 22 and 20 of the first-run reds were plain browser files.
Because the head runs these files 32-wide and the base replay runs them alone, a load red on the head reads as a new red, and per-test red classification lost its signal.
The change
scripts/ci/run-webviews-tests.sh: plain files that drive a real browser (from "playwright",import("playwright")orrequireBrowserLane() run in their own pool,CMUX_WEB_TEST_BROWSER_JOBSwide (default jobs / 4, the same width the dev-server files already use because they run a browser too), beside the other plain files on the remaining jobs. At one job (the base replay) the two pools run one after the other, so the replay still runs one file at a time. Both pools keep the longest-first order.scripts/ci/webviews_test_shards.py: the shard planner counts browser files on their own 8 lanes beside 24 plain lanes. Counted as plain files on 32 lanes they looked free and the split stacked them on one shard (shard 1 of #27329's run held 34 of the 74). Now it spreads them across the shards.- Tests (
tests/test_webviews_tests_per_file.py): the browser pool's width and that jsdom files run beside it, the width override, one job meaning one file at a time, the planner placing browser files beside plain ones, and the split spreading them.
No test assertion, timing limit, timeout, retry budget or red rule changes. No allowlist.
Performance
Same harness both times: the full react-apps-check suite, 4 shards on Blacksmith 32 vCPU, with every webview test file selected (a runner change selects all of them, the same as a base branch run). Before = four base branch runs from today (40858484933, 40858096372, 40867689646, 40868457869). After = this PR's run 40866852261, attempts 1 and 2. "First-run reds" counts the files that failed or timed out on their first run, before any retry.
| metric | before (4 base runs) | after (attempt 1, attempt 2) | delta | harness or run link |
|---|---|---|---|---|
| first-run reds, all shards | 38, 37, 43, 42 | 31, 18 | -40% (mean 40 to 24.5) | react-apps-check (1-4), runs above |
| of those, plain browser files | 22, 20, 24, 24 | 2, 2 | -91% | same; a browser file imports playwright or calls requireBrowserLane |
| reds on the worst shard | 28, 26, 35, 31 | 13, 6 | -68% (mean 30 to 9.5) | same |
| slowest shard job wall | 18.4, 18.5, 20.7, 19.4 min | 20.6, 19.6 min | +0.9 min (mean 19.25 to 20.1) | same |
| new reds (red on head, green on base) | not measured on base (its own runs have no base to compare) | attempt 1: 13; attempt 2: 0 | not comparable | react-apps-base-check |
Most of attempt 1's reds were a dev-server wave on shard 4: 9 composer and gallery files in the 8-wide dev pool, which this PR does not change. That shard ran no browser files, and the same files passed in attempt 2 and on base. Reds from plain browser files dropped from about 22 per run to 2. Attempt 2 passed react-apps-check with no new reds.
Risk
CI-only. The cost is that browser files now run 8 at a time instead of up to 32. The planner accounts for this when it splits, and the slowest shard moved +0.9 min on the means above, well inside the 40 min job timeout. If a shard grows too slow, CMUX_WEB_TEST_BROWSER_JOBS widens the pool without a code change. The timers here are xargs pools that end when their input ends; nothing new keeps running.
Agent session: agent session
š¤ Generated with Claude Code