ci(nightly): republish a verified nightly-next build with the tip's publication tooling

Merged
TE
2d ago
+580-12

Summary

Adds a "republish this verified build with tip tooling" path to the NEXT nightly. Today publish-nightly runs .publish-tooling from github.sha, the build commit, so every fix to publishing needs a new ~1.5 h build before it reaches a build that already passed.

Motivating runs:

  • Run 41815641855 (nightly-next d96836b, build 4181564185501) built, signed, notarized and passed "Verify final nightly DMG", then publish-nightly published nothing. The cause was a supersede bug, since fixed by #35436.
  • The run before it, 41814490749, failed its R2 upload, since fixed by #35337.

With this PR, that build can go out with the fixed tooling and no rebuild:

gh workflow run nightly.yml --ref nightly-next -f nightly_next_republish_run=41815641855 -f nightly_next_republish_attempt=1

The artifacts of that run expire on 2026-11-07 (3-day retention).

How it works

  • New inputs nightly_next_republish_run and nightly_next_republish_attempt, built like nightly_next_unsigned_run and _attempt. decide refuses them in these cases:

    • outside a workflow_dispatch on refs/heads/nightly-next
    • a run or attempt that does not parse
    • combined with any other build or notary mode (continue-only, unsigned resume, fast, build-only, seed, notary tests or probe)
  • A republish builds and submits nothing. It reuses continue_only (now "this run builds and submits nothing"), so every build job skips as it does for a continuation. find-nightly-next-recovery skips a republish. It forces should_build, so publication always runs and the recheck decides. The run title is Republish nightly-next run N attempt M.

  • New job republish-nightly-next has no environment and no secrets, and actions: read only. It runs scripts/ci/republish-verified-nightly.py, which follows resume-unsigned-nightly.py and reuses its downloader. It refuses unless all of these hold:

    • The source is a completed protected nightly-next push of nightly.yml.
    • In the requested attempt, build-sign-notarize-nightly (arm64), Verify signed nightly app (arm64), Verify final nightly DMG (arm64) and build-nightly-app succeeded.
    • await-nightly-tree-marker passed or was skipped.
    • Each artifact (variant, tested digests, SSH daemon, plus the source archive and Chromium pack when present) is live, unique, uploaded by its producer job during that attempt, and downloaded against GitHub's recorded SHA-256.
    • The build commit comes from that build's dSYM receipt, and both it and the source head are ancestors of this run's nightly-next commit.
    • Every DMG matches a digest the source's verify job tested, and the daemon manifest is that build's.

    If all of these hold, it uploads the build in the recovered layout as cmux-nightly-republished-variant-arm64 and -tested-arm64, plus cef-runtime-pack if the source had one.

  • publish-nightly gets a third way to start (needs.republish-nightly-next.result == 'success'). Its "Select the build to publish" step picks the republished build as source=recovered. Every guard then runs unchanged from the tip's .publish-tooling:

    • the lease and the recheck against both feeds
    • "Require every DMG to match a tested digest"
    • the final gate: the writer, the webview verdict read from the source run, known-bad, and the recovered lineage check
    • R2, appcasts and the GitHub release

    The gate's RECOVERED_RUN_ID now comes from the select step's new run_id output, so it names whichever run the published build came from.

  • Same as for a recovered build: no deltas, no release-note signing and no published-in. The legacy-identity jobs are untouched. They read the live NEXT feed, so the next run signs the republished build as cmux NIGHTLY.

Testing

From tests/, python3 -m unittest test_nightly_next_recovery passes all 74 tests (new: RepublishSourceTests, RepublishDecideTests, RepublishWorkflowTests). The new tests cover:

  • validate, for good and untrusted sources, failed or other-attempt verify jobs, the tree marker, producer windows, digests, ambiguity, optional artifacts and the dSYM receipt
  • the DMG and daemon byte checks
  • decide's outputs and refusals, through the existing node decide harness
  • the job's secret-free shape
  • the select step's republished branch and its run_id, and that no kept guard names the republish

Also passing: test_nightly_unsigned_resume, test_nightly_cmux_tui_resolve (the new job is exempt like the recovery jobs: it ships no source), test_nightly_trusted_routing, test_ci_runs_on_fallback, test_nightly_never_blacksmith, test_ci_nightly_off_blacksmith, test_workflow_file_size (421,724 bytes), test_gh_budget_wait, test_nightly_background_steps. Lint: actionlint, plus check-bash32-parse.sh over 423 inline scripts.

Observed / Not run

  • Observed: the tests above. test_nightly_owned_minis_only, test_nightly_next_arm64_only and test_ci_macos_xcode_selection fail with the same failures on base, because they depend on the local environment.
  • Observed: the source run's real jobs and artifacts (/actions/runs/41815641855/attempts/1/jobs, /artifacts) fit the producer windows the validator checks.
  • Not run: the dispatch itself. It happens after merge, at the nightly-next tip.

Changelog

none

Risk: high (release publication path). Merges on an independent SHIP receipt at the exact head.

Agent session: agent session

🤖 Generated with Claude Code


Autofix with [code]smith
Open in [code]smith · Need help on this PR? Tag @codesmith-bot with what you need.


Summary by cubic

Summary

Adds a way to republish an already verified nightly-next build with the tip's publication tooling, so publication fixes reach builds that passed verification without a ~1.5-hour rebuild. Previously publish-nightly used the build commit's .publish-tooling, so fixing a failed publication of a finished build required rebuilding it (run 41815641855 verified build 4181564185501 and then published nothing, a supersede bug fixed by #35436). New dispatch inputs nightly_next_republish_run/nightly_next_republish_attempt make the run build nothing; a secret-free republish-nightly-next job fetches the source run's verified build and publish-nightly publishes it as a recovered build through every existing guard.

Safety

  • The fetch job refuses anything but a completed nightly-next push whose signing and both verify jobs passed in the requested attempt.
  • It checks artifact provenance and each download against GitHub's recorded SHA-256, and requires every DMG to match a digest the source's verify job tested.
  • Like other recovered builds, a republished build gets no deltas, release-note signing, or published-in.