ci: route iOS kit tests through the runner pool
The iOS kit workflow selects the same fixed macOS label on every attempt, so a job can stay queued when that pool starts nothing. This change uses the existing simulator-aware picker to select idle owned capacity first, with Blacksmith macOS as fallback. It accounts for kit runs as one simulator and emits the existing rescue markers. The executing rescue and janitor on main support this workflow through #28431, including its three-minute stall budget and open-PR retry path.
Queue and snapshot reads try the routing token before the job token, with one bounded fallback. Both kit routing-token mint steps now request Actions read access, as the same routing App already does in ci-macos.yml. This reads workflow runs, jobs and capacity-snapshot artifacts through the App quota when the job-token quota is exhausted. No new App, secret, writer scope or fork access is introduced. Production queue improvement is not yet measured.
Regression coverage checks picker names, artifact pins, running and completed simulator accounting, off-fleet placement, quota failures, snapshot fallback, rescue triggers, and the stall budget. Dispatches use the existing routing credential environment.
Performance
| metric | before | after | delta | harness or run link |
|---|---|---|---|---|
| Reported queue wall time | About one hour without a runner | Production improvement not yet verified | Not yet measured | Original incident, attempt 2 |
| Runner-minutes while the job has no runner | 0 | 0 | 0 | Queue time is not runner execution time |
No production speedup is claimed. The earlier picker run still waited about 3h48m after quota exhaustion forced Blacksmith placement. The focused simulated-clock regression verifies an owned kit stall budget of at most 180 seconds. Existing bounded rescue watches are reused; no polling process is added.
Warm resources: this changes placement, not build commands or cache keys. It retains the existing Cargo cache. Python routing checks need no Mac compile or DerivedData. Simulator comparisons should reuse warm DerivedData and module caches; this change does not establish additional persistent Swift caches.
Validation: 248 remote Linux runner-pool and cross-branch rescue-source tests passed at 7d4dc54e in run 41150119507. After merging current base, all 25 runner-routing-table tests pass locally. Focused remote checks will be repeated at the final head, including the routing-table tests. The obsolete kit picker_followup entry added by #28600 is removed while retaining its apple-swift-tests classification. Earlier main-rescue tests passed in run 41096022154. The fleet comparison ran the full CmuxMobileChiefTests target on the same simulator class without a crash: merge base 4543835b had 102 passed and 9 failed; c39d28ef had 103 passed and 8 failed. Every head failure also failed by name at base. Both refs predate #28337's five-second helper rewrite; the newer picker commits do not change Home sources. Current base is now merged, including #28337’s helper rewrite; the earlier comparison remains historical evidence for the previous head. Fresh review remains pending at bf75881c. The original incident run was left untouched.
No capture: CI workflow and picker only
Agent session: agent session
CI red comparison at final head
The automatic cmux-next CI guard group passed at bf75881c3c6ca5016dd2a55718a077b7ec23ad81, covering the runner-pool, owned-pool-source, and routing-table guard group. The tooling-tests job 124403733443 has two base reds: the unwired tests/test_check_objc_selectors_step.py linux-guard registry entry and AppCrateLicenseGate.test_the_ten_first_party_gpl_crates_are_named_exactly with the extra cmux-chats-mirror. Base feat-cmux-next 5f1a9ce9 reproduces both in push run 41159138738, tooling-tests job 124412542698. They are not caused by this PR.
Failure comparison at bf75881c
The remaining red jobs are environmental/base reds, with their causes recorded here:
- The iOS simulator build run 41156282579, job 124415476256 failed before checkout in the fleet hook: it waited for another fleet build for 425 seconds, then refused because that build was waiting for the host. The picker itself succeeded in runner job 124415476487 with
glaeda-std-xcode-26.6,glaeda-ios-sim, andpersistent=true, exactly matching base run 41150994994, runner job 124387387900. This is a fleet admission conflict before any iOS step, not a picker verdict. - The dependent ios-tests job 124423945800 explicitly reports
ios-simulator-build did not pass: failure; it did not run a routing suite and has no independent routing failure. Base's corresponding job 124396711118 passed. - The kit job 124406107026 failed three simulator tests:
TerminalSurfaceFramePacingTests.busyOutputDrawsAtMostOneFramePerDisplayRefresh,TerminalBenchProbeAccountingTests.probeCountsTheResetAndAnIdleEchoShowsAtOnce, andNativeBrowserTabTests.aMacThatForwardsOpensItsLocalhostPageOnThePhone. Each reproduces on base ancestors: frame pacing in base run 41138142557/job 124346936628 and 41147747527/job 124376714398; TerminalBench in 41147747527/job 124376714398; NativeBrowser in 41156807832/job 124408736989, 41153382578/job 124400814436, and 41151767516/job 124390415838. Those heads are ancestors of base5f1a9ce9; these are established base flakes. - The fleet Release compile run 41156282526/job 124406297814 received repeated controller 404s (and two 429s) for the compile result and ended
no resultafter its 40-minute wait. Base run 41159138640/job 124415170265 returnedattempt 1: passwithin seconds. The head failure is a controller result publication outage, not a changed runner-pool path; this PR does not edit the cmux-next compile workflow.
The automatic guard group passed at exact head in job 124403733572; tooling-tests reds are identical on base and documented above. No capture: CI workflow and picker only
Agent session: agent session