ci(web): put required webview shards on fleet first

Draft
TE
4d ago
+34-5

Leo: "WHY ARE TEST RUNS EVEN GETTING F***ING STUCK". The rulebook says our fleet comes first, with Blacksmith only as bounded overflow.

Both webview shard families currently name Blacksmith 16 vCPU directly, so the required base verdict can sit queued behind a stalled paid label. Route head and base shards to the ephemeral 16-vCPU GCP fleet first. Attempt 2 retains the measured 16/32-vCPU overflow and admits at most four shards; later attempts return to fleet. Preserve the eight-shard test split, per-test base comparison, outside-organization route and workflow permissions.

Controller prerequisite: its demand inventory must include top-level callers ci.yml and cmux-next-web.yml; the reusable ci-web.yml filename alone cannot provision runners. Coordinated with the HQ broker routing repair (manaflow-ai/cmuxterm-hq#5762); that registration must land before activating this route.

Prior art: #29033 measured 16 vCPU against 32 and set the overflow pair; #29666 made eight shards; #28511 keeps same-test base flakes advisory. This retains those semantics. pr-overlap found no overlapping open PR; manual title search found no matching fleet-web routing implementation.

Warm sources remain the Bun and Playwright Actions caches and the existing bundle cache keys in .github/warm-sources.json. These jobs need Linux browser dependencies; no Swift compile or shared Mac cache is added.

Validation: fleet workflow-routing and existing web validation tests pending. Regression evaluates both shard jobs with the overflow switch set/unset, attempts 1–4, concurrency bounds and outside-organization routing.

Queue finding: queue_janitor.find_orphans does not implement a three-minute no-start rescue. It needs newer same-label jobs to have started, or the 24-hour hard age, so a label with zero starts is not promptly moved. Routing fixes new attempts; the missing generic rescue is tracked in manaflow-ai/cmuxterm-hq#5751, which was read before this change. This PR does not claim an automatic 180-second JIT start guarantee. The existing fleet reconciler takes 5–18 minutes between some observed successful runs. Web shards have no nightly publishing siblings, and this change cancels no run.

Performance

MetricBeforeAfterDeltaHarness or source
Blacksmith shard runner-minutes/day, head + base111,677.9 on 2026-10-30 (9,043 jobs)Not observed before deploymentNot measuredFleet summary runtime_by_job.day, summed n Ɨ mean / 60 for both shard families

Agent session: agent session
šŸ¤– Generated with Claude Code


View with [code]smith Autofix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need.


Summary by cubic

Routes the head and base webview verdict shards to the ephemeral 16-vCPU GCP fleet first so a stalled Blacksmith label can no longer hold up the required base check.

  • Attempt 2 is the bounded paid overflow (at most four shards, 16/32 vCPU per the existing CI_CLOUD_OVERFLOW_SAVED switch); attempts 3+ return to fleet.
  • Preserves the eight-shard test split, per-test base comparison, outside-organization hosted route, and timeout.
  • Adds validation covering attempts 1–4, the overflow switch set/unset, concurrency bounds, and outside-organization routing.

Migration

  • The fleet controller must register the top-level callers ci.yml and cmux-next-web.yml; the reusable ci-web.yml filename alone won't provision runners. Coordinate with the HQ routing repair in manaflow-ai/cmuxterm-hq#5762, which must land before this route activates.
  • The missing queue_janitor no-start rescue is out of scope, tracked in manaflow-ai/cmuxterm-hq#5751.