ci: the overflow watch's runner-pool reconcile reads as cmux-bot (~325/h off GITHUB_TOKEN)

Merged
TE
2d ago
+18-1

The overflow watch's "Point the runner pools at live members" step reads with the repository GITHUB_TOKEN. That pool is 5,000 requests an hour, every workflow shares it, and it runs out every hour right now. The step now reads with the cmux-bot read-only token that the same job already mints for "Probe and switch" (#35316).

What changes

  • GH_TOKEN for the step becomes ${{ steps.cmux-bot-token.outputs.token || github.token }}. The step only reads with it:

    • the linux spill's queue sample: in-flight runs listed by age slice, plus each sampled run's jobs;
    • the pool snapshot artifact that runner_pools.py reads (list plus download).

    The cmux-bot token is permission-actions: read, which covers both.

  • Variable writes stay on SWITCH_TOKEN (the route App). If the cmux-bot mint fails, the expression falls back to the workflow token, so the step behaves as before.

  • New test: tests/test_ci_runner_pools.py::test_the_reconciler_reads_as_cmux_bot_not_the_shared_workflow_token.

  • No new secret, App or permission. The mint step and its actions: read scope are unchanged.

Measurement (2026-11-04, 14:03–15:02Z)

A job on a measurement branch read GET /rate_limit with its GITHUB_TOKEN every 2 s for an hour (that endpoint is free). I then compared the pool's increments with each watch run's step boundaries, taken from the job logs. The fit was joint with nightly-next-feed-watch, whose runs overlap these in time:

stepGITHUB_TOKEN requests per runruns/hper hour
Point the runner pools at live members (before)54 ± 56~325
same step (after, projected)0, moved to cmux-bot60
Probe and switch, mints, checkouts~0 (already on cmux-bot since #35316)6~0

The whole pool measured 4,824/h over the same hour. cmux-bot's own pool limit is 12,500/h; it used about 5,300/h over the same window, so it has room for ~325/h more.

Verification

  • python3 -m unittest tests.test_ci_runner_pools: 47 OK
  • python3 -m unittest tests.test_ci_cloud_overflow_switch: 67 OK
  • actionlint on the workflow: clean

🤖 Generated with Claude Code

Agent session: agent session


Autofix with [code]smith
Open in [code]smith · Need help on this PR? Tag @codesmith-bot with what you need.


Summary by cubic

Moves the overflow watch's runner-pool reconcile step off the shared GITHUB_TOKEN so its reads stop burning ~325 requests/h of the 5,000/h pool that currently runs out every hour.

The step's GH_TOKEN now uses the cmux-bot read token the same job already mints for "Probe and switch", which covers all the step's reads (the spill's queue sample and the pool snapshot artifact). A failed mint falls back to github.token, and variable writes stay on SWITCH_TOKEN. Adds a test asserting the token swap and that the mint runs before the reconcile step.

Summary by CodeRabbit

  • Chores
    • Updated automated runner-pool checks to use an available app token, with a workflow-token fallback.
    • Added checks to verify token permissions and fallback behavior.