ci: CI routers and report steps take hosted Ubuntu first as declared light jobs
What
ci.yml's changes job (the run's router: change classification and the runner picks) read vars.LINUX_RUNNER first. That variable holds one Blacksmith size, blacksmith-4vcpu-ubuntu-2404 since 02:15Z, so every pull request's checks waited behind that one queue. On 2026-11-02, run 41385632466's job 125086959342 sat unassigned (runner_id 0) on the 4 vCPU label from 04:00:13Z to 04:04:26Z, behind about 721 queued jobs on that size, with a pull request's required webview verdict waiting behind it.
changes runs before any picker job could choose a runner for it, so it is a router of the kind the light Linux exception covers (resolvers, decide/report steps, pickers). It is now declared in .github/runners.json hosted_light_jobs and takes:
- attempt 1: GitHub-hosted
ubuntu-24.04, which is ephemeral like Blacksmith. The job mintsGLAEDA_ROUTE_APP_KEYtokens, so it never goes to a persistent fleet label. Forks already run this job onubuntu-24.04. - a re-run: the reconciled ephemeral
LINUX_RUNNERmirror, bounded to Blacksmith labels, then 8 vCPU (16 while the overflow switch is on, pertests/test_ci_blacksmith_linux_failover.py). - fork pull requests: their existing Blacksmith branch, unchanged.
No live fleet Linux runner can take it today (cmux-fleet-linux-x64 has no runners), and nothing runs before the router that could check one.
The same change covers cmux-next-fast-gates.yml's resolve (names the head and base, about 10 s of work) and post (one commit-status write). On 2026-11-02 cmux#31754's post sat on RUNNER_POOLS.linux's saturated 4 vCPU size. Neither job holds a secret, so a re-run takes the reconciled pool.
scripts/ci/runner_routing_table.py's hosted-light policy, the central allowlist that tests/test_ci_self_hosted_guard.sh reads, gains three narrow allowances:
- an
environmentexpression whose only possible values are'ci-routing-agent'or''. That is the router's existing expression, and any other value is still refused. - an 8 vCPU retry size alongside 16 and 32. A light job should not need 16 or 32.
- the overflow switch's
vars.CI_CLOUD_OVERFLOW_SAVED && '<paid size>'just before the retry size. - a fork pull request's Blacksmith branch before hosted Ubuntu (like the fork-owner branch), and the bounded
contains(fromJSON('["blacksmith-…"]'), vars.LINUX_RUNNER) && vars.LINUX_RUNNERmirror as the retry. A barevars.LINUX_RUNNERor a non-Blacksmith label in that list is still refused. fromJSON(vars.RUNNER_POOLS).linuxas the retry, only for a job that references no secret.
Environments, secrets, permissions and runner trust are unchanged.
Performance
| metric | before | after | delta | harness or run link |
|---|---|---|---|---|
changes queue wait, run 41385632466 | 253 s (unassigned on 4 vCPU) | hosted start, typically seconds | about -250 s per affected run | job 125086959342 timestamps |
changes work time | 11 s | 11 s | 0 | same job |
Blacksmith minutes/day for changes (whole-minute billing) | about 2,500 (one per CI run; about 2,500 ci.yml runs in the 24 h to 04:00Z) | re-runs only | about -2,500 | Actions runs API count for ci.yml |
fast gates post queue wait, run 41389051872 | queued on 4 vCPU behind about 430 jobs | hosted start | (wait was still growing when measured) | fleet dashboard queued, 04:40Z |
Validation
Ran locally on Linux:
- passed:
tests/test_ci_self_hosted_guard.sh,test_runner_routing_table,test_ci_runner_set,test_ci_blacksmith_linux_failover,test_ci_blacksmith_linux_sizes,test_ci_fork_runner_routing,test_ci_linux_single_size,test_ci_change_areas(234 passed),test_ci_owned_pool_rescue,test_ci_owned_pool_sources,test_ci_pr_runner_pool,test_ci_runner_capability_resolver,test_ci_label_events_skip_full_ci,test_ci_delta_since_green,test_ci_manual_dispatch_guard_workflow,test_cmux_next_fast_gates,test_ci_overflow_pick_reads_poolandtest_ci_runner_pool_reconcile. That is 74 test files that touch these workflows or the runner tables: 2570 passed. test_ci_workflow_run_sources::test_ci_failfast_keeps_failure_rollups_and_bounds_observed_tailsfails the same way on base.test_ci_warm_sourcesfails only onnightly.ymljobs this PR does not touch. That failure is already on base.actionlint: nothing new.
Follow-ups (not here)
ci-web.yml#web-subarea-scopeandci.yml#ghosttykit-release-checkreadfromJSON(vars.RUNNER_POOLS).linux, which also resolves to the saturated 4 vCPU size right now.web-subarea-scoperestores anactions/cacheentry, so it needs a check that the cache still hits off Blacksmith before it moves to hosted.
Agent session: agent session
🤖 Generated with Claude Code