#25490 ci(publish): gh_app_token.sh and the uploaders' release-App-first fallback (tooling, inert)

Approved by Leo on 2026-10-26 (the secrets/permissions change this PR held for): "for all those waiting on you things go ahead but just follow lawrence for now i havent seen much yet".

Rebased onto feat-cmux-next at de54c57d59a (squashed), resolving a conflict with #25470 in publish-release-assets.py: the request loop and asset_digest keep #25470's rate-limit backoff and add the App-token switch before it. Two digest tests now patch the redirect-safe opener. The re-review found two gaps, fixed in a follow-up commit: finalize-release.py download now falls back to github.token when the App token is refused, and gh_app_token.sh reads the job ledger fail-closed and reserves each wait before sleeping, as gh_budget_wait.sh does. Codex gpt-6-astra xhigh: SHIP at the final head.

Part 1 of 2 for the cmux NEXT nightly publish path (nightly.yml on nightly-next). This PR adds only the tooling, and it does nothing until a job passes GH_APP_TOKEN or installs the wrapper. Part 2 does that wiring and holds every secrets, environment and permissions change. Both parts were held for the same approval.

Why. Run 40298250989 had already built and signed the nightly when its upload POSTs were refused with HTTP 403 "API rate limit exceeded for installation", from 12:04:51Z to about 12:14Z. During those refusals, core still read 4899/5000, and the 403s carried no x-ratelimit-* headers. So the hourly core budget was not spent: this was GitHub's secondary limit on bursts per actor (or possibly a per-repository cap). Every workflow's GITHUB_TOKEN counts as that one actor. The release App is a separate actor with its own limits.

What changed (no workflow permissions:, environment: or secrets change in this PR)

• New scripts/ci/gh_app_token.sh --install puts a gh on PATH. A call whose GH_TOKEN is github.token goes out with the App token first; a call that passes any other token is left alone.
  • When the App token is refused (401, 403, 429, a rate limit, or GraphQL's "Resource not accessible"), the same call is sent again with github.token. A refused request did nothing on GitHub's side, so this is safe for writes.
  • "Resource not accessible" marks that API area (actions, issues, graphql, …) as one the App can't reach; the rest of the job sends it straight to github.token.
  • A 401 drops the App token for the job.
  • /rate_limit reads keep github.token, since that is the budget gh_budget_wait.sh waits on.
• The fallback backs off and retries both tokens when both are refused for a limit, or with a 403 that is not a permission refusal: Retry-After when shown, otherwise 60 s, 120 s, 240 s. The waits go into gh_budget_wait.sh's per-job ledger and are capped by GH_BUDGET_WAIT_MAX_SECONDS. A 403 with no headers backs off; it does not fail fast.
• publish-release-assets.py, finalize-release.py and prune_nightly_release_assets.py read GH_APP_TOKEN first and switch to GH_TOKEN for the rest of the run on a 401/403/429.
  • The switch is thread-safe: the upload workers share one client, and a concurrent refusal switches once.
  • Hashing an asset with a null digest goes through the same fallback, and no longer sends the token across the asset API's redirect to the storage host.
• tests/test_gh_app_token.py (new, registered in ci-guards release-notary and tests/test-execution.toml) drives the wrapper against a fake gh, including the gh_budget_wait.sh interplay, and tests the uploaders' switching, concurrency, redirect and prune fallback. test_github_release_publication.py now patches the opener.

Tests (on a cmux-ci Linux box): test_gh_app_token.py, test_github_release_publication.py, test_prune_nightly_release_assets.py and test_gh_budget_wait.py pass.

Risk: normal on its own (inert). #25470's uploader backoff is on the base now; the App-token switch runs first, and a rate-limited refusal after it backs off as #25470 does.

Agent session: agent session

🤖 Generated with Claude Code

---

^{Need help on this PR? Tag @codesmith-bot with what you need.}