#33028 release signer: wait out Apple notary rate limits instead of failing the nightly

Nightly run 41571567978 (the tip build Leo's PRs were waiting on) failed at signer-app. Apple refused the Computer Use helper's notarization:

"code": "RATE_LIMIT", "status": "429", "title": "Too many notarizations",
"detail": "Exceeded daily limit for notarizations. Try again in 0:04:11."

The signer treated this like a rejection and failed the run, so nothing published. Apple only asked for four minutes.

Change

• scripts/ci/release-signer/signer-env.sh gets notary_submit, a wrapper around notary-submit. On a 429 RATE_LIMIT it waits the time Apple names plus 15 s, then submits again. All waits share one budget, CMUX_NOTARY_RATE_LIMIT_SECONDS, 600 s by default, which fits inside the signer jobs' 40/75 min timeouts beside their notary waits.

  • Apple keeps nothing from a refused submission, so the retry duplicates nothing.
  • Any other failure still fails at once.

• sign-app.sh (the helper submission), notarize-app.sh and notarize-dmg.sh submit through it. Both notarizations stay, and so does the stapled app inside the DMG.

• tests/test_release_signer_scripts.py: the fake signer can now answer with Apple's 429, and a stand-in sleep records each wait. The new tests cover:

  • one refusal, then success (waits 266 s, submits twice);
  • a wait longer than the budget (no wait, fails);
  • the budget spent across several refusals;
  • a failure that is not a rate limit (no retry).

  The three rate-limit tests fail without the change.

Waiting adds time only when Apple has already refused, a case that failed the whole nightly before.

Follow-up, not in this PR: we are near Apple's daily cap. Each nightly now notarizes the helper, app and DMG for two identities (since #32140). Skipping app-identical promotions (#32689) cuts the count; the Mac fallback scripts (notarize-computer-use-helper.sh, notarize-nightly-dmg.sh) do not wait on a 429 yet.

Agent session: agent session

🤖 Generated with Claude Code

---

^{Open in [code]smith · Need help on this PR? Tag @codesmith-bot with what you need.}

---

Summary by cubic

The release signer now waits out Apple's notary rate limits instead of failing the nightly. Previously a 429 RATE_LIMIT response aborted the run even when Apple only asked for minutes.

• Adds notary_submit in signer-env.sh, wrapping notary-submit: on a 429 it waits the time Apple names plus 15 s and submits again, within CMUX_NOTARY_RATE_LIMIT_SECONDS (600 s by default).
• All three notarization paths (helper, app, DMG) now submit through it. Refused submissions leave nothing at Apple, so retries are safe; other failures still fail immediately.
• Adds tests covering a single rate limit, budget exhaustion across refusals, and non-rate-limit failures.