#25404 ci(nightly): a newer nightly builds and signs alongside a waiting one, never replacing it

Two nightly concurrency groups let a newer run cancel a waiting nightly, and the nightly rule forbids that: "a newer tip builds alongside a running one instead of waiting behind it; no evicting concurrency group on those workflows; a CI test enforces it."

Groups changed

• build-nightly-app used nightly-next-build-app-warm. Every attempt-1 nightly-next push on the owned route shared one slot: one build ran, one waited, and a newer push replaced the waiting one.
  • On 2026-10-26, holders sat queued on blacksmith-12vcpu-macos-26, which served nothing from about 01:20Z (see #25351), and newer promotions waited behind them.
  • The slot existed because of FIFO queueing on one warm host (2026-10-19). The route is now a pool (vars.CI_SIDE_LANE_RUNNER, then Blacksmith), so that reason is gone.
  • An older run whose nightly-next has moved on already skips its build (resolve-nightly-cmux-tui-client's superseded step), so per-run slots don't build stale heads.
• sign-nightly-identity-legacy used nightly-next-legacy-identity, a shared group that evicted pending signers.
  • Both groups are now per-run.
  • The sign job's "Skip a build the old feed already has" step and the publish job's order check keep the old feed right. Two runs may submit the same build to Apple, which only costs Apple time (cx-vws1x, already accepted).

Groups kept shared

These build nothing, so replacing a pending one drops only an older run's write. Each is listed in the test with this reason:

• the promotion request groups: nightly-next-promote and nightly-next-promote-ack;
• the feed writers: release-publish-<tag> and nightly-next-legacy-identity-publish.

Test

• tests/test_nightly_delivery_concurrency.py gains test_nightly_builds_and_signers_never_share_an_evicting_group. It fails when any nightly.yml job group lacks github.run_id and isn't one of the listed promotion or writer groups. It also requires the workflow-level group to be per-run on nightly-next.
• The file gets a __main__ block, and ci-guards.yml runs it in the release-notary group. Before this it was unwired: test_ci_workflow_guards_are_wired listed it.
• cmux-next-dev-daily.yml is not covered yet. Its publish and tip groups belong to the DEV owner, and the test extends there once they are fixed.

Not in this PR

• Why the tick promoted an old commit: on 2026-10-26 it promoted a142865 (21:03 PT on 10-25). That commit predates #25351, so its nightly.yml still routed to the dead 12vcpu label. It was the newest commit with a published tree and a green Release compile while push compiles were stuck on the gateway (fixed in #25360).
  • Every promotable commit from now on is after #25351, so this gap closes on its own. There is no new guard.
  • With per-run slots, a newer promotion is no longer blocked behind such a run.
• The classic main nightly keeps its workflow-level nightly-shared group, which lives in main's copy of nightly.yml. That one is a separate PR to main.

Risk

• Medium-low.
• Two nightly-next builds can run at once, using one more pool slot or Blacksmith slot.
• Publication stays ordered by the existing checks (nightly_version.py check-build, the publish-time recheck, and the legacy order step).

Tests (local)

• Pass: test_nightly_delivery_concurrency, test_nightly_trusted_routing, test_ci_nightly_off_blacksmith, test_nightly_unsigned_resume, test_nightly_universal_build.sh.
• Base red: test_ci_workflow_guards_are_wired now lists only the pre-existing test_cmux_next_guardrails.py.

Agent session: agent session

🤖 Generated with Claude Code

---

^{Need help on this PR? Tag @codesmith-bot with what you need.}