ci(janitor): nightly notarization follow-ups reserve their Blacksmith macOS label

Merged
TE
7d ago
+83-6

What

The queue janitor's macos-pool-load snapshot counts a queued macOS job as reserved_queued only when its run's workflow is freshness work. The cmux-next placement (scripts/ci/next_macos_cascade.py) keeps PR overflow off a Blacksmith macOS label while that count is above 0. Until now the reserved set was a regex matching only release.yml and nightly.yml, so .github/workflows/auto-resume-nightly-notarization.yml ("Continue nightly notarization") didn't count. On 2026-10-30 at 12:43Z its three wait-and-staple jobs queued on blacksmith-6vcpu-macos-26 behind 22 PR overflow jobs, and nothing kept the label free for them.

Change

  • scripts/ci/queue_janitor.py: the regex RESERVED_POOL_WORKFLOW is replaced by reserves_pool(path). It returns true for release.yml (the declared RESERVED_RELEASE_WORKFLOWS) and for every delivery workflow in scripts/ci/delivery_workflows.py's existing DELIVERY_GLOBS. That file is already the one list the janitor uses to decide which runs it must never cancel. Matching still uses the workflow path only, so a CI run whose name contains release=arm64 still doesn't reserve a label.
  • Workflows that now reserve and didn't before, each with a Blacksmith macOS job: auto-resume-nightly-notarization.yml, resume-nightly-notarization.yml, nightly-next-canary.yml, nightly-next-launch-smoke.yml, nightly-next-train.yml and cmux-next-dev-daily.yml (DEV verify and tip DMG/deltas; the cascade already counts DEV's Release compile as freshness work in FRESH_JOBS).
  • Not reserved, on purpose: the iOS TestFlight and App Store workflows (they're iOS publishing, not a macOS feed's freshness), and the DEV source build of cmux-next.yml (matching is by path only, and that path is the PR workflow).

Tests

New ReservedPoolWorkflowTests in tests/test_ci_queue_janitor.py:

  • A queued auto-resume-nightly-notarization.yml job on blacksmith-6vcpu-macos-26 counts as reserved_queued (the same holds for resume-nightly-notarization.yml).
  • An invariant over the real .github/workflows: every workflow whose file name or name: marks it as nightly, release, promote or notarization work, and that has a job whose runs-on can resolve to a Blacksmith macOS label (a literal blacksmith-*macos* or a MACOS_RUNNER* variable), must reserve its label. The only exceptions are listed with reasons: iroh-release-gate.yml, a manual test gate, and tmux-corpus.yml, a manual corpus test titled "terminal nightly". A new nightly or notarization workflow can't be missed.
  • The PR workflows ci.yml, cmux-next.yml and ci-macos.yml don't reserve (a run name containing release=arm64 is used).

Evidence, run locally:

  • Base 4ae385f8 with the new tests: 7 failures (0 != 1 for auto-resume, resume, nightly-next canary, launch smoke and train).
  • Head: tests/test_ci_queue_janitor.py 251 OK, tests/test_ci_pr_runner_pool.py 252 OK, tests/test_ci_simple_pool_picker.py 46 OK, tests/test_delivery_workflows_cancel.py 25 OK, tests/test_nightly_delivery_concurrency.py PASS.

feat-cmux-next

feat-cmux-next has an older copy of queue_janitor.py with the same regex. It doesn't need this change: the janitor runs only on schedule, workflow_run (CI requested) and workflow_dispatch, and the first two always run from the default branch. The snapshot every branch's placement reads is built by main's copy. The feat copy will pick this up the next time main is merged into it.

Risk: low. When a nightly, DEV or notarization job is queued on a Blacksmith macOS label, PR overflow now waits instead of taking that label. This is what the cascade already does for nightly.yml and release.yml.

Agent session: agent session

šŸ¤– Generated with Claude Code

View with [code]smith Autofix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need.

Summary by cubic

Fixes the queue janitor's macos-pool-load snapshot so nightly notarization follow-ups and other delivery workflows reserve their Blacksmith macOS labels. Previously only release.yml and nightly.yml reserved; on 2026-10-30 auto-resume-nightly-notarization.yml's wait-and-staple jobs queued behind 22 PR overflow jobs on blacksmith-6vcpu-macos-26 because their workflow wasn't in the reserved set.

  • Replaces the RESERVED_POOL_WORKFLOW regex with reserves_pool(), which returns true for release.yml and every workflow in delivery_workflows.py's DELIVERY_GLOBS (nightly, its verify/notarization follow-ups, DEV, and promote). Workflows are still matched by path only.
  • Adds ReservedPoolWorkflowTests that fail on any workflow named or filed as nightly, release, promote, or notarization with a Blacksmith macOS job that isn't either reserved or explicitly exempted (e.g., iroh-release-gate.yml, tmux-corpus.yml).

Summary by CodeRabbit

  • Bug Fixes
    • CI capacity tracking now accounts for queued jobs across release, nightly, development, promotion, and notarization workflows, while continuing to exclude designated workflows. This gives a more accurate view of available build capacity and helps prevent queued work from being overlooked when the system tracks resource availability.