#26657 ci(delivery): a nightly that can never deliver may be cancelled (latest-wins, known-bad list)

Leo, 2026-10-26: "SKIP THE OLD NIGHTLY BUILDS THEN WE'RE JUST BUILDING FROM THE TIP".

The agents' cancel guard reads scripts/ci/delivery_workflows.py from main. It refused stale nightly-next runs 40628145226 (head d7cf72ee), 40624308094 and 40623450445 with "not certain to fail: its publish job may still run". Those runs could never deliver. Each head contains 4b5c86ed (Home first-run NSStackView crash at launch) and lacks its fix 1daea603. Each is also an ancestor of a newer nightly-next tip that has its own run.

What changes

• Latest-wins (never_publishes): a nightly-next run counts as skippable even when its publish job hasn't run, in either of two cases:
  • the feed's release already records a commit that contains the run's head;
  • the branch tip is a newer commit (shown by ancestry) whose own run is queued, in progress or succeeded.
    The feed table is LATEST_WINS_FEEDS (nightly.yml on nightly-next → release nightly-next).
• Known-bad list (scripts/ci/nightly_known_bad.json, known_bad): entries are {bad_since, fixed_by, reason, filed_by}, with full SHAs; fixed_by is null until a fix lands. A nightly run whose head contains bad_since and lacks fixed_by is certain to fail. The list is seeded with the launch crash above. A malformed list counts as unread.
• Re-runs: superseded_unstarted now judges a re-run on its own attempt's jobs, so a queued attempt 2 counts as not started even when attempt 1 built. A re-run of a build that ran is a new build: until a job of this attempt starts, nothing of it has started, and the newer-head check still has to show the same build. A job whose attempt can't be read refuses.
• Picker jobs: pick-*-runner jobs (pick-nightly-app-runner, pick-nightly-sign-runner) only choose a label. \bapp\b and sign no longer read them as the build.
• Anything unread (the tip, the feed, an ancestry, the list) never allows a cancel. The queue janitor still skips only runs that never started; the two new rules are read by the agents' guard only.

Tests

tests/test_delivery_workflows_cancel.py (15 tests, added to ci-guards) covers:

• a queued attempt 1;
• a queued attempt 2 after an attempt 1 that never built, and after one that built;
• a started build in this attempt (refused);
• unread job attempts (refused);
• a picker job that ran while the build never did;
• the feed or tip newer, and the kept cases: tip is its head, no tip run, failed tip run, unread tip or ancestry;
• the known-bad schema and seed, fixed or unread heads, and non-nightly workflows;
• a recorded replay of the three live runs: all allowed by both rules.

The existing janitor, delivery-concurrency and feed-watch suites pass on the remote fast-checks box: 347 passed.

A dry run of the updated guard against live GitHub on these three runs says ALLOWED for each. The two that have since completed were judged as the in_progress runs they were when refused.

Before: refused ("not certain to fail: its publish job may still run"). After: allowed ("its head d7cf72ee5655 contains 4b5c86ed0aef (Home first-run NSStackView crash at launch) and lacks its fix 1daea6030cb1, so it is certain to fail"). With an empty known-bad list, all three are allowed through latest-wins instead ("the newer commit 39f472b834f3 is promoted at the tip of nightly-next and run 40631810292 is done with it").

Risk: this widens which started nightly runs an agent may cancel. Both rules fail closed on anything unread, and latest-wins also refuses when a PR is open at the run's head (that check lives in the guard). No run is cancelled by this change.

Agent session: agent session

🤖 Generated with Claude Code

---

^{Need help on this PR? Tag @codesmith-bot with what you need.}

---

Summary by cubic

Allows the agents' cancel guard to cancel nightly-next runs that can never deliver, fixing three stale runs it wrongly refused. The guard previously refused runs 40628145226, 40624308094 and 40623450445 with "not certain to fail: its publish job may still run", even though each head contains a known launch crash and lacks its fix.

• Latest-wins: a nightly-next run counts as skippable even before its publish job runs, when the feed's release already records a commit containing the run's head, or when the branch tip is a newer commit with its own run queued, in progress, or succeeded, provided the tip run shows the same build: each build job of this run's current attempt that is not skipped is listed there, live or succeeded.
• Known-bad list (scripts/ci/nightly_known_bad.json): a run whose head contains bad_since and lacks fixed_by is certain to fail. Seeded with the Home first-run NSStackView crash at launch.
• Re-runs are judged on their own attempt's jobs: a queued attempt 2 counts as not started even when attempt 1 built, a tip re-run only shows a build done in its own attempt, and the queue janitor now reads a re-run's own attempt too. Picker jobs (pick-*-runner) only choose a label and are no longer read as the build.
• Anything unread (the tip, the feed, an ancestry, the list) never allows a cancel.

Adds 15 tests covering these cases. A dry run against live GitHub on the three previously refused runs now says ALLOWED for each.

Summary by CodeRabbit

• Bug Fixes

  • Nightly delivery runs are canceled when a newer run for the same build supersedes them, their commits have already been published, or they contain an unfixed known-bad commit.
  • Cancellation checks use the current run attempt and exclude runner-picker jobs from build-job checks.
  • Incomplete ancestry or missing, malformed, or unconfirmed data does not by itself trigger cancellation.

• Tests

  • Added coverage for cancellation decisions, including attempt handling, latest-run checks, published commits, and known-bad entries.