ci(cla-rerun): re-run CLA checks just after the budget reset; guard rejects only with API-failure evidence; cover forks
Follow-up to #25326 (the scheduled CLA quota re-run).
What was wrong
On 2026-10-26 from about 08:40Z, CLA Assistant and CLA policy guard were red on every open pull request into main (#25403, #25406, #25407, #25408, #25411, and later #25172 and #25390). The GITHUB_TOKEN installation budget was spent. The 08:57Z sweep logged 0 of 5000 left (floor 1000) and skipped, which is correct. The gaps were these:
- Guard rejects were re-run blindly. Every CLA policy guard failure was re-run until attempt 8, with no evidence. The guard hides every
PolicyError(validate-cla-policy.rb), so a real policy rejection got 7 re-runs too. - Forks were never covered. GitHub lists no
pull_requestson a fork's run, socurrent_headwas always false for them (#25411). - Timing. The budget resets hourly (about :13 past on 2026-10-26), and push-triggered workflows drain it again within minutes. The
*/10cron really fires every 25 to 40 minutes, so a pass usually landed while the budget was empty again. - The base was never filtered. That part was fine: main pull requests were already covered.
Change
- One planned pass per reset. When the budget is low and the reset is under 15 minutes away, the pass sleeps until 20 s after the reset, reads
rate_limitonce more and runs then. This is not a loop. A farther reset is left to a later cron. - Its own workflow.
.github/workflows/ci-cla-quota-rerun.ymlreplaces thecla-quota-rerunjob in ci-queue-janitor.yml. It has cron*/10, concurrency groupci-cla-quota-rerunwith cancel-in-progress false, and a job timeout of 22 min. A pass waiting for the reset never holds the macOS sweep's concurrency group. A dispatch defaults to a dry run. - Floor. It drops from 20% to 10% (minimum 500). Re-runs are capped at
(remaining - floor) / 40, reserving 40 GITHUB_TOKEN calls per re-run, so the pass never drains the budget itself. Right after a reset the floor never blocks. - Coverage. Open pull requests into
mainandfeat-cmux-nextare read per base (up to 5 pages each, with a warning past that). They are mapped by head SHA, so forks count. A same-repository run whose own pull request is into another base is left alone. The App token getspull-requests: read. - A guard reject is re-run only with evidence of an API failure:
- (a) its own log shows
rate limit,HTTP 403or403 Forbiddenbefore the reject line, or - (b) a CLA Assistant job, on any pull request, logged
API rate limit exceeded for installationwithin 15 minutes of it. The budget is the repository's and both workflows share it. A secondary or user rate limit is not evidence.
- (a) its own log shows
- Any other reject is never re-run. It gets a
::warning::that names it as possibly a real policy rejection. - Case (b) is needed. The guard's own log never shows the API error, because it prints only the generic line. Every guard log from this outage looks the same.
cla.yml, cla-policy-guard.yml, validate-cla-policy.rb and the supply-chain tests are unchanged. A re-run changes no policy: the same trusted workflow runs again on the same head.
Risk
Low. The job is continue-on-error, scheduled or dry-run dispatch, and its only write is rerun-failed-jobs. Compared with before, it re-runs fewer guards (only with evidence) and more forks. It holds a Linux runner for up to 15 minutes before a near reset.
Tests
tests/test_cla_quota_rerun.py has 20 cases. They cover:
- main and feat-cmux-next bases, with other bases skipped
- a run whose own pull request is into another base
- per-base reads
- a fork matched by its head
- a guard's own 403
- a 403 after the reject, which is not evidence
- a bare reject: never re-run, warned
- installation-limit evidence inside and outside the window
- a secondary limit, which is not evidence
- one pass just after a near reset, a low budget after the reset, a far reset that is not waited for
- the floor cap
They passed on cmux-ci, along with unittest discover -p 'test_ci_*.py' (2345 tests) and check_workflow_concurrency.py.
Agent session: agent session
🤖 Generated with Claude Code
Summary by CodeRabbit
- CI Improvements
- CLA checks retry qualifying rate-limit and API-failure errors for open pull requests targeting
mainorfeat-cmux-next, including pull requests from forks. - Retries are limited to recent failures on the latest pull request revision and subject to an attempt cap. Guard rejections without API-failure evidence are not retried.
- Reruns run through a scheduled or manually triggered workflow; manual runs include a dry-run option.
- Reruns respect API request limits, briefly waiting for a budget reset when appropriate and adjusting retry capacity to available budget.
- CLA checks retry qualifying rate-limit and API-failure errors for open pull requests targeting