#25885 ci(nightly-next): prepare the nightly variant off the Macs, byte-identical

Leo: "Why are the minis tied up?" / on byte identity: "and we proved that it's all byte identical?"

What changes

prepare-nightly-variant (primary and fallback) now runs off the Macs. Primary uses blacksmith-4vcpu-ubuntu-2404 (ubuntu-24.04 outside manaflow-ai). Fallback uses ubuntu-24.04: it does nothing after a successful primary and runs the full preparation after a failed one. There, prepare does only work that is proven byte-identical:

• llvm-lipo (llvm-18) as lipo, for the thin and the arch checks. A small shim trims the trailing space llvm-lipo -archs prints.
• The license-tree and libintl checks, and packaging into cmux-prepared-app.tar.gz. The artifact name and the tar layout are unchanged.

Everything that runs or rewrites Mach-O moves to Mac jobs that already run:

• build-nightly-app gets "Bundle the cmux-tui client", placed after "Strip unsigned nightly app before transfer". It runs install-cmux-tui-client.sh (including the remote-probe --json and acpmux --version probes), strip-release-bundle.sh, write-cmux-tui-version.sh and check-daemon-capabilities.sh --static-only. This adds about 30–40 s to a job that is on a Mac anyway.
• build-nightly-ghostty-cli-helper gets "Strip the Ghostty CLI helper" (strip -S -x), the strip prepare used to apply to the injected helper. llvm-strip does not give Apple strip's bytes, so it stays on the Mac.
• verify-signed-nightly gets "Run bundled Ghostty theme picker helper regression", one isolated hunk after the existing step. That step already runs the sidecar check (verify-diff-sidecar-artifact.sh --require-signed) on the signed app, so prepare's unsigned copy of the check is dropped.

prepare now starts with "Require the cmux-tui client build-nightly-app bundled". It refuses an unsigned app whose cmux-tui.version is missing or doesn't match the resolved commit.

This frees 3–6 min of owned minis per nightly. Recent primary prepare times on owned minis were 4.6, 2.9, 3.0, 5.9 and 5.5 min, with outliers of 12.8 and 21.4 min under queue pressure. After this change prepare uses 0 Mac minutes, and build-nightly-app gains about 30–40 s.

Byte identity

Test sha: 8b3b1891681798cabadc45b37a9d1d5192b437b7, from nightly run 40380339203 (arm64).

• Old: the cmux-nightly-prepared-app-arm64-8b3b189 artifact that run uploaded.
• New: the same run's unsigned app and helper, put through the new path. On a Mac with Xcode 26.6 and that sha's scripts, I ran the build-nightly-app client step and the helper strip. With llvm-18 lipo, I ran the prepare half: inject the helper, inject licenses, drop CEF, thin, run the libintl check and tar.
• Comparison: every path, its type, mode, symlink target and file sha256 were hashed into one tree hash:

paths: old 8553, new 8553; files 6792, links 22
tree hash old c233fb332d43b86d14885533d59e1975ba6a4b19ee02047327ccba21f756e418
tree hash new c233fb332d43b86d14885533d59e1975ba6a4b19ee02047327ccba21f756e418
differing paths: 0

Apple strip -S -x gives the same helper bytes with Xcode 26.6 and 27.1. The Mac-stripped helper (6ffdf502…) is byte-equal to bin/ghostty in the old prepared app.

Tests

• test_ci_nightly_off_blacksmith.py checks several things:
  • both prepare jobs run off the Macs;
  • neither prepare job calls any install, strip, probe, theme-picker, sidecar, select-ci-xcode, ditto, otool or codesign step;
  • build-nightly-app runs Strip, then Bundle, then Archive;
  • verify-signed-nightly runs the theme picker and the sidecar check;
  • the helper job strips.
• test_nightly_trusted_routing.py: the prepare-routing test now expects the same runner on every attempt.
• .github/nightly-step-coverage.json has entries for the moved and new steps.
• I ran the 69 test files that read nightly.yml through cmux-ci test. The failures left are the 6 that fail the same way on base: test_ci_change_areas (react-apps aggregate), test_ci_fork_runner_routing x2, test_ci_workflow_path_filter_parity, test_ci_workflow_run_sources and test_runner_label_policy (every runner variable reported).

Risk

• Unsigned resume of an old run: an app saved before this change has no bundled client. prepare now refuses it with a clear message instead of preparing it.
• Theme picker: the regression now runs on the signed app in verify-signed-nightly, not on the unsigned one in prepare. A regression now fails verify and blocks publish, as it did before.
• Fallback: the fallback now runs on ubuntu-24.04, and its only job is preparation.
• Overlap: #25845 also edits prepare's runs-on (its Mac fallback label). Once prepare is off the Macs, that hunk no longer applies to prepare. Whichever PR lands second rebases.
• No new secrets or permissions.

Agent session: agent session

🤖 Generated with Claude Code

---

^{Need help on this PR? Tag @codesmith-bot with what you need.}