nightly-next: new builds go to the Dev ring first and move to Nightly on a canary pass

Merged
TE
16d ago
+586-14

Picking Dev in cmux NEXT's Update Channel does nothing today: every item on the cmux NEXT feed is tagged cmux-next, which every ring accepts. With this change and repository variable NIGHTLY_NEXT_DEV_RING=on, each new nightly-next build goes out as cmux-next-dev first, so only apps on Dev get it right away (same app, com.cmuxterm.app.nightly.cmux-next, updating in place). It moves to cmux-next (Nightly) once the canary passes it. Unset or off keeps every item on cmux-next, which is today's behavior, so merging this changes nothing until someone sets the variable.

How a build moves from Dev to Nightly

  • Published: build-sign-notarize-nightly, the recovery path and generate-nightly-deltas tag new items cmux-next-dev when the variable is on.
  • Qualifies: the durable canary verdict verdicts/<build>.json is pass. It's the signal that already exists, and the train already uses it for its base. I didn't add a soak timer: nothing in the workflows has one, and a timer would promote builds the canary never saw.
  • Promoted: nightly-next-verdict.yml sets every feed item's channel from the verdicts after it writes each verdict. The write is If-Match on the bytes it read, like the rollback. republish-nightly-deltas, which lists earlier builds again, tags them from the same verdicts before publishing. The verdict documents are the only source of truth, so any later writer corrects a missed promotion. Turning the variable off puts every item back on cmux-next at the next verdict or publication.
  • Old feed: legacy-identity-target (the cmux NIGHTLY-identity feed for pre-NEXT installs) copies only builds already on cmux-next.
  • Logic: all of it lives in scripts/ci/nightly_next_ring.py (channel, retag, select).

Old builds never see an unpromoted item

Builds from before rings allow only cmux-next: UpdateFeedResolver.Channel.allowedSparkleChannels is allowedSparkleChannels(ring: .nightly), and UpdateDriver+SPUUpdaterDelegate.allowedChannels(for:) returns it to Sparkle. Sparkle drops an item whose sparkle:channel isn't in allowedChannels, and the app's own model of this (AppcastItem.isOffered(allowing:) on the release-rings branch: an untagged item is always offered, a tagged one only when allowed) works the same way. Main's builds allow no channel at all. The new test test_nightly_and_older_builds_never_see_an_unpromoted_build checks that rule for Dev, Nightly, pre-ring and main updaters on a retagged feed.

The guard

nightly-next-guard.py appcast used to require exactly [cmux-next]. It now requires exactly one tag per item, from the four ring tags (cmux-next-stable, cmux-next, cmux-next-canary, cmux-next-dev). An untagged item, a second tag or a foreign tag is still refused. A test checks that the guard, nightly_next_ring.py and UpdateRing.swift/UpdateFeedResolver.swift name the same four tags. Nothing publishes canary or stable items yet.

How much faster Dev is

Not much, in the best case:

  • Build cadence: builds landed about every 21 minutes on 2026-10-21 (30 builds, 11:59 to 22:09 UTC).
  • Canary latency: the canary runs at :03/:18/:33/:48 (15 minutes apart) and has a 25-minute job timeout. The one durable verdict so far came 9 minutes after its build published. So promotion trails publication by about 10 to 40 minutes.
  • App check interval: the app checks hourly (UpdateSettings.scheduledCheckInterval = 60 * 60). For a Mac checking hourly, Dev is ahead of Nightly by less than one check interval.
  • Builds the canary skips: the canary checks only the newest build at each tick. A build that's superseded before its tick never gets a verdict and stays on Dev; Nightly moves on with the next build that passes.

Right now, on would freeze Nightly. The only verdict written so far is a fail (build 3968001295301, 22:18 UTC). Every scheduled canary run I opened that actually ran the canary failed (6 of them between 17:25 and 22:13 UTC); the others skipped already-checked builds. That's why the switch defaults to off and the doc says to turn it on only while the canary passes.

A Nightly app can be offered a promoted build only while that build is on the feed. The first publication writes a single-item feed with only the new build, and the delta republish adds up to 5 earlier builds back. Between those two writes, Nightly is offered nothing new. That's safe, but it's a delay.

Risk

risk:high: release/updater publication. Mitigations:

  • Default off: with the variable unset, the only changes are a guard that accepts four tags instead of one, a no-op select on the old feed, and an extra feed read in the verdict job that writes only if a channel differs (it won't).
  • When on, the failure direction is safe: an unreadable verdict counts as none, so the item stays on Dev. Nightly is never offered a build early; at worst it gets one late.
  • Concurrency: the feed write in the verdict job is If-Match. When it loses a race it writes nothing, and the next writer derives the same channels from the verdicts.
  • Tooling version: republish-nightly-deltas now runs the guard and the ring script from the workflow commit (.publish-tooling, as publish-nightly already does), because the build ref can predate them.
  • Not covered by this PR: a promoted build can drop off the feed (see above). The legacy feed waits while nothing is promoted.

Changelog

none

Observed / Not run

Observed, on c169198818787ac851ad894874fcc2381c3ba053:

  • python3 tests/test_nightly_next_ring.py: 17 tests OK. The workflow test for the verdict promotion was red before the verdict workflow change.
  • python3 tests/test_nightly_next_guard.py: PASS. Against the base guard it fails on the three new ring-tag cases and the tag-sync check (red on base, green here).
  • These neighbors pass: test_nightly_next_feed_rollback, test_nightly_next_recovery, test_nightly_next_post_publish, test_nightly_next_train, test_nightly_republish_delta_guard, test_ci_test_execution_registry, test_ci_workflow_guards_are_wired, test_ci_run_guards, test_ci_linux_guard_routing, test_cmux_next_guards_route, test_feat_ci_guard_group, test_pr_lane_coverage, test_ci_actionlint_covers_every_workflow, test_ci_workflow_run_sources, test_ci_reusable_workflow_permissions, test_ci_release_product_reuse.
  • actionlint on the three changed workflows: the only finding is base's 17-input workflow_dispatch on nightly.yml, which is the same on base. Runner-label noise is ignored.
  • python3 scripts/cmux-next/prepush-gate.py: no new violations.
  • The live feed, verdict documents and canary run history quoted above were read on 2026-10-21.

Not run:

  • No nightly or verdict workflow was dispatched, so the retag and promotion steps haven't run against the real feed or R2.
  • test_ci_guard_workflow_structure.py and test_ci_change_areas.py failed only because files were missing from my sparse checkout (agent-chat/package.json, cmux.xcodeproj). CI runs them.
  • No app build. The Swift claims come from reading the code.

Agent session: huge-blue

🤖 Generated with Claude Code


Summary by cubic

Picking Dev in cmux NEXT's Update Channel does nothing today: all items on the feed are tagged cmux-next, which every ring accepts. With this change, setting repository variable NIGHTLY_NEXT_DEV_RING=on publishes each new nightly-next build as cmux-next-dev (Dev ring only) and moves it to cmux-next (Nightly) once its durable canary verdict is pass. Unset or off keeps every item on cmux-next, so merging this changes nothing until someone enables the variable.

  • Promotion happens in nightly-next-verdict.yml (retags the live feed with If-Match) and in republish-nightly-deltas; the verdict documents are the single source of truth.
  • Nightly and pre-ring builds accept only cmux-next, so they never see an unpromoted build; the old cmux NIGHTLY-identity feed copies only Nightly items.
  • The feed guard now requires exactly one of the four ring tags per item instead of exactly cmux-next; a test keeps the guard, the ring script, and UpdateRing.swift in sync.
  • With on, Nightly gets nothing new while the canary passes no build, so the variable should be enabled only while the canary is green.