ci(nightly, dev): publish gh calls go out as the release App first, github.token as fallback

Merged
TE
10d ago
+283-5

Approved by Leo on 2026-10-26 (the secrets/permissions change this PR held for): "for all those waiting on you things go ahead but just follow lawrence for now i havent seen much yet".

Rebased onto main after #25491 merged (squashed; the same patch the review passed). main's #25398 already put cmux-next-dev-daily publish in environment: release with no concurrency group (R2 lease and feed compare-and-swap) and a 75-minute timeout, so this PR no longer adds that environment or changes the timeout; the conflict kept main's job header. Codex gpt-6-astra xhigh: SHIP at the rebased head (restacked unchanged onto the parent's review fix).

Part 2 of 2 for main: the workflow wiring. The classic NIGHTLY publisher (main's nightly.yml) and cmux NEXT DEV's publisher (cmux-next-dev-daily.yml, which runs main's copy on workflow_dispatch) now send their gh calls as the release GitHub App first, with github.token as the fallback. The tooling is part 1, #25491. This PR is stacked on it and gets retargeted to main once #25491 lands. The NEXT nightly path is #25490 and #25477.

Why. Run 40298250989 was refused with HTTP 403 "API rate limit exceeded for installation" from 12:04:51Z to about 12:14Z. During those refusals, core still read 4899/5000, and the 403 responses carried no x-ratelimit-* headers. So the hourly core budget was not spent. This was GitHub's secondary limit on bursts per actor (or possibly a per-repository cap). Every workflow's GITHUB_TOKEN counts as that one actor. The release App is a separate actor with its own limits.

What changed

  • Each job below mints a contents-only App token (continue-on-error, 2 min) and installs scripts/ci/gh_app_token.sh from a sparse checkout at the workflow commit (.gh-tooling, persist-credentials: false). The DEV checkout uses main (default_branch).
  • The uploader, prune and finalize steps of publish-nightly, and the uploader in republish-nightly-deltas, get GH_APP_TOKEN. They run from .gh-tooling (the workflow commit), because the build checkout may predate App-token support.
  • No job here installs gh_budget_wait.sh, so the wrapper waits alone, with a 10-minute cap (GH_BUDGET_WAIT_MAX_SECONDS=600).

Fallback. A failed mint (continue-on-error) installs nothing, and the job runs on github.token exactly as today. If both tokens are refused for a limit, or with a 403 that is not a permission refusal, the call backs off and retries both: Retry-After when shown, otherwise 60 s, 120 s, 240 s. A 403 with no headers backs off; it does not fail fast. On main the cap is 10 minutes per job (GH_BUDGET_WAIT_MAX_SECONDS=600), which fits cmux-next-dev-daily publish's existing 75-minute job and 45-minute publish step.

Permissions / environment / secrets diff (file by file)

  • .github/workflows/nightly.yml
    • secrets.CMUX_RELEASE_APP_ID / secrets.CMUX_RELEASE_APP_KEY (already in release; no new secret names) are read by a new mint step in three jobs:
      • publish-nightly: token permission-contents: write
      • generate-nightly-deltas: read
      • republish-nightly-deltas: write
    • These jobs were already in environment: release. No permissions: or environment: change.
  • .github/workflows/cmux-next-dev-daily.yml
    • publish already runs in environment: release on main (#25398), which holds the App credentials; no environment: change here.
    • publish reads secrets.CMUX_RELEASE_APP_ID/_KEY; its token gets permission-contents: write, the job's existing level.
    • A sparse checkout of main's scripts/ci/gh_app_token.sh is added (persist-credentials: false). The job still runs no code from the app.
    • timeout-minutes is unchanged (75, from main). permissions: is unchanged. No concurrency group is added (#25552's test passes).
  • .github/workflows/ci-guards.yml: a new step runs tests/test_publish_app_token_workflow.py. No permission changes.
  • Other files: tests/test-execution.toml.

Not moved, and why

  • The promote jobs (promote-nightly-next, -acked) already make their one write, the nightly-next move, with the release App. Their reads (runs, check runs, deployments) need Actions/Checks read, which the App lacks, so they stay on github.token. Moving them would mean widening the App's permissions, which is a separate ask.
  • cmux-next-dev-daily sign reads Actions run/artifact APIs, which the App can't reach either.
  • build-nightly-app's attestation needs attestations/id-token, which the App does not have.

Tests (cmux-ci Linux): tests/test_publish_app_token_workflow.py (new, in ci-guards), test_gh_app_token.py, test_github_release_publication.py, test_cmux_next_dev_daily_dispatch.py, test_cmux_next_dev_tip_feed.py, test_nightly_delivery_concurrency.py, test_ci_quota_hold_gate.py and the test_nightly_* workflow tests all pass.

Risk: high (secrets reach the App token in 4 jobs). A failed mint leaves each job exactly as it is today.

Agent session: agent session

🤖 Generated with Claude Code

Summary by CodeRabbit

  • Improvements
    • Improved reliability of nightly and development build publishing, including release assets and update metadata.
    • Publishing can fall back to existing authentication if preferred credentials are unavailable.