ci(nightly-next): main mirror of the release train (#22661): promote guard G1'/G2', tick, canary verdict

Merged
TE
16d ago
+1530-14

Main mirror of #22661 (PRs 1 and 2 of the nightly train design, leo-workspace#624). It has the same shape as #21900 and #22652.

Why main

Main runs the promote job and guard, the tick (workflow_run), the canary schedule, and the dispatch registration of new workflows. Until these files are on main, #22661 changes nothing.

What it mirrors

  • scripts/ci/promote-nightly-next.mjs: the train path. G1' needs the second parent on feat-cmux-next, every revert in that parent's history, and the tree rebuilt by nightly_next_train.py verify. G2' needs a green nightly-next train Release compile from a nightly-train/* dispatch run of this exact SHA. The first parent must be the current nightly-next. Plain commits keep G1/G2.
  • nightly.yml: both decide steps pass verifyTrain, and the sparse checkout adds the train script.
  • scripts/ci/nightly_next_train.py: the guard runs main's copy, never the SHA under test.
  • nightly-next-tick.yml: starts the train. With NIGHTLY_NEXT_TRAIN unset (dry-run) it only plans, and the plain promotion still runs. With on, the train replaces the promotion.
  • nightly-next-canary.yml: a verdict job dispatches nightly-next-verdict.yml on nightly-next. It only warns until nightly-next carries that workflow.
  • nightly-next-train.yml and nightly-next-verdict.yml: dispatch registration only. Runs on main do nothing.
  • .github/release-surface.txt: lists both new workflows.

Verification

  • Ran: the guard's G1'/G2' cases in tests/test_nightly_promotion_admission.py (66 cases ran locally; the yaml ones run on CI), and actionlint on the changed workflows.
  • The train's own tests (33 cases) live in #22661.
  • Behavior with the variable unset: the tick also dispatches a plan run, the canary dispatches a verdict write, and promotion is unchanged.

Base merge and guard fix (25bab407)

  • I merged main d6239486 cleanly. After the merge the tick still holds #23033's refusal tally inside the promotion step.

  • On main, two ci-guards checks were red for this PR's own reasons (run 39527541864). Both are fixed:

    • Runner label policy: ci-health-report.yml and ci-repo-variables.yml now report NIGHTLY_NEXT_TRAIN_RUNNER.
    • Signing secrets: the drive job in nightly-next-train.yml reads CMUX_RELEASE_APP_* in environment nightly-next-train. tests/test_ci_signing_secrets_need_release_environment.py now allows that one reader in that one environment, and only for the release App credentials, the same way it allows the promote jobs.
  • Ran locally: both guards, test_ci_repo_variable_defaults, test_nightly_promotion_admission (66 cases) and scripts/ci/guards-local.sh. That last run passed 122 steps. Its 2 failures came from my machine, not this branch: local actionlint 1.7.12 flags if: false in main's nightly.yml too (CI pins 1.7.7), and a local git fetch crashed in pack-objects during the delta-CI test.

  • feat-cmux-next has the same signing-guard red. #22661 brought it in there, and a separate feat PR has to fix it.

  • ea58d4f7 copies #23299's fix of the review's medium finding. The drive job in nightly-next-train.yml now checks out github.sha, never inputs.tip. Before the release App token is minted, it refuses any tip that isn't 40 hex or isn't an ancestor of the dispatched feat-cmux-next head. tests/test_nightly_next_train_drive_pin.py checks this. Main's copy stays identical to feat's.

Risk: risk:high (release train, promote guards, signing-secret guard).

Leo's calls

  • Setting NIGHTLY_NEXT_TRAIN=on. It also needs environment nightly-next-train (deployment branch feat-cmux-next) holding CMUX_RELEASE_APP_ID/CMUX_RELEASE_APP_KEY, because GITHUB_TOKEN can't push train commits that change workflow files.
  • The switch is one-way. Once a train commit is on nightly-next, plain promotions are refused, so keep on.

🤖 Generated with Claude Code

Agent session: cosmic-gray


View with [code]smith Autofix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need.


Summary by cubic

Main mirror of #22661 (the nightly-next release train): moves the train's promote guard, tick, and canary schedule onto main, where they actually run.

  • A train commit promotes only when its tree rebuilds as its second parent minus its listed reverts (main's copy of scripts/ci/nightly_next_train.py) and a nightly-next train Release compile job for that SHA on a nightly-train/* ref passed. The compile runs on hosted macos-26 (override NIGHTLY_NEXT_TRAIN_RUNNER), never Blacksmith.
  • The tick dispatches nightly-next-train.yml in dry-run by default; NIGHTLY_NEXT_TRAIN=on replaces the plain promotion, and the switch is one-way, so keep it on.
  • The canary's verdict job dispatches nightly-next-verdict.yml on nightly-next for the durable verdict and only warns until that branch carries the workflow.
  • The drive job checks out the dispatched feat-cmux-next head, never inputs.tip, refusing a tip that isn't 40 hex or an ancestor before minting the release App token; require_on_tip also refuses a base or carried-car commit outside the tip's history, keeping the module identical to feat-cmux-next's.
  • nightly-next-train.yml and nightly-next-verdict.yml do nothing on main; .github/release-surface.txt lists both. The release App read needs environment nightly-next-train (deployment branch feat-cmux-next) holding CMUX_RELEASE_APP_ID/CMUX_RELEASE_APP_KEY; the two ci-guards reds (runner label policy, signing secrets) are fixed.
  • Uses risk: high: release train, promote guards, signing-secret guard.

Summary by CodeRabbit

  • New Features

    • Added an automated nightly-next release train with planning, candidate builds, verification, bisection, and controlled promotion.
    • Added durable canary verdict recording and validation for nightly-next builds.
    • Added optional scheduled train dispatches with configurable planning or execution modes.
    • Added compile checks and artifact reporting for train builds.
  • Bug Fixes

    • Prevented promotion when train commits, ancestry, build checks, or verification results are invalid or outdated.
  • Tests

    • Expanded coverage for train admission, promotion, and workflow safety scenarios.