ci(janitor): carry the gates' own state per glaeda label in macos-pool-load

Merged
TE
7d ago
+163-1

Why

GitHub lists a runner whose glaeda gate holds its listener off as offline, the same as a dead one. The overflow route reads GitHub's runner list, so it cannot tell "this label is full" from "this label is broken", and the two disagree with the gates. The fleet dashboard now keeps the gates' own state in the CI_FLEET_GATES repository variable (manaflow-ai/cmuxterm-hq#5321). This PR puts that state in the macos-pool-load artifact the pickers already download.

What

  • The janitor's sweep step gets FLEET_GATES: ${{ vars.CI_FLEET_GATES }} through env only. A variable read costs no API request, and the value is never expanded into a script.
  • fleet_gates() in scripts/ci/queue_janitor.py checks the reading. Any field out of shape refuses the whole reading, which is the safe side because readers treat missing as full. It keeps only the documented fields and writes them into the snapshot as gates. With no reading it writes gates_error instead.
  • Per glaeda label: listening, busy, held_offline {total, units, cpu, livelock, other}, down, unreported, queued, oldest_queued_s, as_of, plus a top-level as_of. The meaning of each field and the staleness rule are documented beside fleet_gates() and in the workflow header. A reading over 5 min old is stale, and a reader treats stale or missing as full for jobs queued over 3 min. Readers judge as_of against their own clock, never the snapshot's generated_at.

The readers' use of gates (the pickers' "label is full" decision) is a separate change, owned by the overflow-route lane.

Tests

FleetGatesTests in tests/test_ci_queue_janitor.py has 5 tests: the fields kept per label, refused shapes (bad JSON, version, as_of, negative or bool counts, a missing reason, a non-glaeda label), a sweep writing gates, a sweep writing gates_error, and the workflow passing the variable through env only. All 5 fail on main (red) and pass here (green). The full test_ci_queue_janitor.py suite passes, and so do the other tests that read the janitor (change areas, fork routing, health report, owned-pool rescue, warm state, pr_runner_pool).

Risk

The change is additive: one new snapshot key. pr_runner_pool.py and the other readers ignore keys they do not know.

Agent session: agent session

šŸ¤– Generated with Claude Code