ci(queue janitor): page in-flight runs to their total_count, not to the first short page
Summary
The queue janitor lists in-flight runs one status at a time (status=queued, in_progress, pending) and stopped at the first page with fewer than 100 runs. GitHub's status-filtered run listing can return a short page before the last one. At 06:10Z on 2026-10-26, status=queued returned 99 runs on page 1 and 85 on page 2, against a total_count of 182. So each sweep dropped every queued run older than page 1:
- The pool snapshot undercounted the oldest waits. The 05:57Z
macos-pool-loadputblacksmith-12vcpu-macos-26's oldest wait at 24 minutes. Run 40259152940's fast tier had been queued there for 205 minutes by then.next_macos_cascade.pyandpr_runner_pool.pyroute by those numbers, and the cascade's stall check reads that oldest wait. - Old runs were never acted on. Category b never cancelled run 40259152940, even though #25328 (merged at 6f07da41) made the janitor list cmux-next runs and PR #24021 had landed at 04:03Z. The 05:57Z sweep cancelled eight newer cmux-next stale runs and never saw this one. Its job finally got a Blacksmith runner at 06:08Z.
The fix: paging continues until a page is empty or the runs read reach that status's total_count. When there is no total_count, it falls back to the short-page rule. MAX_RUN_PAGES (3) still caps it, so a sweep costs at most one more request per status than before. owned_pool_rescue.GitHub.failed_runs() (status=failure) gets the same rule.
Risk: low. More runs are visible to the janitor, so each category may act on older runs it used to miss. Those runs were always in scope; the janitor just never saw them.
Testing
pytest -q tests/test_ci_queue_janitor.py tests/test_ci_owned_pool_rescue.py -k "InFlightRunListing or short_page" on cmux15 (Python 3.13):
- base: 2 failed (
test_a_short_first_page_does_not_hide_the_older_queued_runs,test_failed_runs_reads_past_a_short_page_while_the_total_says_more). - head: all pass. Across the janitor, rescue, starved-label and picker suites: 659 passed, 121 subtests passed.
Observed / Not run
- Observed: the page sizes and
total_countabove, from the live API. The 05:57Z snapshot. That sweep's log (eight stale-pr cancels, no mention of 40259152940). Both test runs. - Not run: a live sweep with this code.
Changelog
none
Agent session: cosmic-gray
š¤ Generated with Claude Code
![View with [code]smith](https://pr-comments-assets.blacksmith.sh/codesmith/view-with-codesmith-dark-v3-3x.png)
![Autofix with [code]smith](https://pr-comments-assets.blacksmith.sh/codesmith/autofix-with-codesmith-dark-3x.png)
Need help on this PR? Tag @codesmith-bot with what you need.
Summary by CodeRabbit
- Bug Fixes
- CI run listings now continue through short pages when more results may be available. Duplicate runs are listed only once, and listings stop when a page is empty or the page limit is reached.
- Tests
- Added coverage for pagination across short and empty pages, duplicate runs, page limits, and listings without a reported result count.