ci(cmux-next): no GitHub-hosted Macs: drop the hosted overflow and seed pool, x86_64 checks under Rosetta, and a guard

Merged
TE
10d ago
+353-235

Leo, 2026-10-27: "THE GITHUB HOSTED MACS / ARE THE SLOWEST THINGS / THEY PREVIOUSLY WREN'T EVEN UNDER CONSIDERATINO"

The routing order is Linux, then a free owned Mac, then Blacksmith macOS as the last resort. GitHub-hosted macOS is not in it, not even as a fallback. This removes every route on feat-cmux-next that still reached a hosted Mac, and adds a guard so none comes back.

1. fast-tier seed and the cascade overflow

  • scripts/ci/next_macos_cascade.py: drops HOSTED, HOSTED_CAPACITY and HOSTED_JOBS. fast-tier and swift-canary no longer overflow to macos-26 when Blacksmith is full or stalled. A job Blacksmith can't take now stays on the owned label, where an idle mini or the rescue takes it. The placement no longer writes hosted_jobs or hosted_runner.
  • .github/workflows/cmux-next.yml: fast-tier and swift-canary no longer read the hosted route (in runs-on and CMUX_NEXT_RUNNER), and the 45-minute "cold hosted build" timeout is gone (both back to 20).
  • .github/workflows/cmux-next-fast-tier-seed.yml: SEED_POOLS keeps only the Blacksmith pool. The hosted pool only warmed the hosted overflow, which is gone.

2. x86_64 run-time checks run under Rosetta
cmux-tui-artifacts.yml macos-cross-runtime-x86_64 and macos-cross-parity.yml runtime-x86_64 ran on macos-15-intel, because no provider but GitHub offers an Intel macOS image.

  • What they check: each Linux-built x86_64 Mach-O has the same load commands, dylibs, signature state and version text as the Mac-built one, and the Linux-built cmux-tui passes smoke-remote-release.sh.
  • Where they run now: vars.MACOS_RUNNER_26 || 'blacksmith-6vcpu-macos-26' (fork branch unchanged).
  • How: a step makes sure Rosetta 2 runs x86_64, installing it with sudo -n if it is missing. The check script then runs under arch -x86_64, so uname -m reports x86_64 and both the Linux-built and Mac-built binaries are executed translated.
  • Signal kept: every comparison is unchanged.
  • Signal lost: real Intel-silicon execution. It is the same trade scripts/smoke-launch-macos-app.sh already makes.

Both jobs stay compare-only. The artifacts job is continue-on-error, and the parity workflow runs on schedule or dispatch. Nothing is exempted. Intel coverage of the shipped app on main's ci-macos-compat.yml is a separate PR.

3. Guard: tests/test_ci_no_github_hosted_macos.py
It fails on a hosted macos-* label in three places:

  • any workflow or composite action line: runs-on, matrix rows, pool lists, defaults;
  • any quoted label in scripts/ or .github/actions, since a script's output feeds runs-on (the cascade's old HOSTED = "macos-26");
  • the manaflow-ai fleet in .github/runners.json.

It allows only the fork branch (github.repository_owner != 'manaflow-ai' && '<label>'); a hosted_runner matrix row is not exempt. Any other exemption must be an exact line with a reason, and must still exist. The one exemption is an example string in resolve_runners.py's error message, which selects no runner. The guard is wired into ci-guards.yml and tests/test-execution.toml (linux-guard).

On base it fails with four hits: the seed pool, the cascade HOSTED, and the two macos-15-intel lines. On this head it passes. test_ci_self_hosted_guard.sh drops its now-dead exceptions for those lines.

Hosted-Mac minutes moved (24 h audit, 2026-10-27)

JobBeforeAfter
fast-tier seed, hosted pool21 jobs, 891 min/day0
fast-tier / swift-canary overflow1 job, 10 min/day0
x86_64 run-time checks (macos-15-intel)0 seen in 24 h (runs only when the Mac builder publishes, or daily/dispatch)0, on Blacksmith

Takes back #22501: that PR added the GitHub-hosted macos-26 overflow for fast-tier and swift-canary and its guard exception; Leo's 2026-10-27 order quoted above removes hosted Macs from the routing order, so this reverts that route on purpose.

Risks

  • When Blacksmith is full, PR fast-tier and canary jobs wait on the owned side queue instead of starting cold on a hosted Mac.
  • If a Blacksmith macOS image lacks Rosetta and passwordless sudo, the x86_64 check fails red. The job is compare-only, so nothing else is blocked.

Tests (local, Python only): these pass:

  • test_ci_no_github_hosted_macos.py
  • test_ci_next_macos_cascade.py (75)
  • test_cmux_next_checks_workflow.py (158)
  • test_ci_side_lane_placement.py
  • test_ci_macos_cross_shadow_jobs.py
  • test_ci_nightly_no_hosted_mac.py
  • test_ci_linux_compile_runners.py
  • test_ci_runner_capability_resolver.py

These fail exactly as on base, with no new failures: test_ci_fork_runner_routing.py, test_runner_label_policy.py, test_runner_routing_table.py, and test_ci_self_hosted_guard.sh (chief-evals.yml, release smoke).

Agent session: agent session

šŸ¤– Generated with Claude Code