ci: shard the webviews suite twelve wide

Merged
TE
17d ago
+50-11

Summary

react-apps-check runs the webviews suite as 2 file shards on 8 vCPU Blacksmith runners. Over recent green runs, shard 1 took a p50 of 15.5 min (17.8 max) and shard 2 took 13.2 min. Nearly all of that time is one phase. run-webviews-tests.sh runs about 50 dev-server test files per shard, two at a time, at about 28 s each, which takes about 12 min. The other 120-odd files finish inside a minute.

This PR raises the matrix to 12 shards, so each shard holds 4-5 dev-server files. A concurrency probe ran 256 Blacksmith jobs at 32 vCPU at once with a p90 queue of 21 s, so 12 jobs here won't queue.

  • .github/workflows/ci-web.yml: 12 include entries with shard_count: 12. PR-scoped rules still run on shard 1 only.
  • tests/test_web_validation.py: requires at least 12 shards, numbered 1..N, all with the same shard_count.

Expected cost: the wall time on the critical path goes from about 16 to about 5 min. Runner minutes go from about 29 to about 60 per run, because each shard repeats about 1.5 min of setup (checkout, bun install, Playwright, bundle build). That is about $0.46 → $0.96 per run at $0.016/min for 8 vCPU. I'll post the measured numbers from this PR's own run.

Fork PRs run the same matrix on hosted ubuntu-24.04.

Testing

  • python3 -m unittest -k shard_wide test_web_validation (from tests/) failed on 1b116ff8: 2 not greater than or equal to 12. It passes on the head.
  • test_web_validation, test_pr_red_checks_comment and test_webviews_tests_per_file pass.
  • test_ci_change_areas.py has the same 34 local failures with and without this change (local environment).
  • actionlint is clean.

Observed / Not run

  • Observed: the tests above, actionlint, and the baseline timings from recent cmux-next-web runs.
  • Not run yet: the 12-shard job itself. This PR's CI run is the measurement.

Changelog

none

View with [code]smith Autofix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need.

Summary by cubic

Shards the webviews test suite onto four 32 vCPU runners so react-apps-check finishes in about 5 minutes instead of 14–16.

The old two-shard, 8 vCPU setup spent about 12 of its minutes running dev-server test files two at a time. Twelve 8 vCPU shards barely helped (14.3 min) because each shard started with a cold Vite cache warm-up. Four 32 vCPU shards share one warm cache across about 33 dev-server files per shard and run eight files at once. The shard-independent steps (lint, type check, React Compiler check, eager JS budget, merge-driver test) now run only on shard 1, so a lint failure shows as one red check instead of four.

  • Updates the CI matrix to 4 shards on blacksmith-32vcpu-ubuntu-2404 and adds a validation test requiring 4 numbered, consistent shards with those shard-independent steps gated to shard 1.

Review: SHIP at dfcbc83125c9 (cmux-review subagent; correctness first)
Not UI: CI matrix and step gating only; no app or webview code changes.

🤖 Generated with Claude Code