ci: the routing table fails a macOS job pinned to one fixed label

Merged
TE
6d ago
+478-54

Leo, 2026-10-28: "Bro, I thought we already had logic to automatically route stuff. How can we still have pins like this? I hate pins. Like, just pin the set of the stuff that isn't hilariously slow."

cmux-next-ios-kit.yml's kit-tests runs on
... || vars.CI_PAID_MACOS_OVERFLOW == '1' && vars.MACOS_RUNNER_26 || 'blacksmith-6vcpu-macos-26': one fixed Blacksmith macOS label, with no owned Mac first and no stall-aware failover. None of the runs-on checks caught it:

  • tests/test_ci_runs_on_fallback.py refuses only single-host labels (a glaeda-runner-* label or a mini's name) and the 12 vCPU Blacksmith Mac as a last resort.
  • scripts/ci/runs_on_values.py refuses only runner-name variables that are read bare.
  • scripts/ci/runner_routing_table.py counted only a bare vars.CI_SIDE_LANE_RUNNER / CI_LIGHT_LANE_RUNNER as bypassing placement.

A Blacksmith 6 vCPU macOS literal is none of those, so a job that moved off the owned pool onto a fixed Blacksmith label passed every check.

Change. runner_routing_table.check() now also fails a macOS job whose runs-on (plus its matrix, when it reads one) reads no picker output. A picker output is a needs.<job>.outputs.<x> other than the static capability map, or a caller's inputs.*. Each failure names the pickers to use: ios_runner_pool.py, simple_pool_picker.py or side_lane_placement.py.

The 42 macOS jobs pinned this way today each get a picker_followup in .github/runners.json. Like linux_followup and placement_followup, a followup that no longer applies fails, so the list only shrinks. The release-signing jobs are marked as needing a signing picker (per-job keychain, exclusive claim). A placement bypass already has its placement_followup and is not counted twice.

Red/green.

  • The new fixture test test_a_macos_job_on_a_fixed_label_fails fails on the exact kit-tests expression.
  • Before kit-tests got its followup, the tree test test_the_tree_classifies_every_macos_job failed and named cmux-next-ios-kit.yml#kit-tests runs on macOS from a fixed label.
  • kit-tests now carries a picker_followup that points at #28145, so this guard can land now and refuse new pins. #28145 routes the job through ios_runner_pool.py: owned simulator Macs first, Blacksmith on retry, with the owned-pool rescue watch, whose main-side half #28431 has merged. With #28145's workflow overlaid on this tree, the check reports kit-tests no longer runs on a fixed label; remove its picker_followup, so #28145 drops that one line when it lands.

New fixture tests cover:

  • the exact kit-tests expression;
  • a bare literal, a variable plus a literal, and the capability map;
  • a literal matrix against a picked matrix;
  • a picker output and a caller's input;
  • a stale followup;
  • no double count with a placement bypass.

Validation: tests/test_runner_routing_table.py, test_ci_runner_capability_resolver.py, test_runs_on_labels.py, test_ci_no_github_hosted_macos.py and test_feat_ci_guard_group.py were run locally. test_ci_change_areas.py has one failure (test_repository_helpers_route_by_where_they_run), which is the same on base. No workflow file changes.

No capture: CI routing check only.

Agent session: agent session

šŸ¤– Generated with Claude Code