ci: keep cmux-next fast tier off dead side queues

TE
6d ago
+996-188

The fast tier could fall through to an unavailable side label when placement found no idle Mac. It now uses a current-attempt owned allocation or the Blacksmith cascade. Bot attempt 2 honors its freshly allocated owned runner. GUI and session routes also require an online runner.

The shared label_full_or_dead adapter reads per-label gates from the pool artifact or CI_FLEET_GATES, checks each label timestamp, and detects saturated owned pools, stale/missing readings after a three-minute wait, Blacksmith cap/backlog, and 30-minute no-start outages. The cascade preserves queued nightly work's priority. fleet_jobs, gateway_runner, and the client filter remain available for the separate fleet-client implementation; full Mac jobs retain their Mac routes.

Rescue discovers unmarked side-lane runs, recovers missing cmux-next PR associations, and detects queued jobs even when a sibling failed. Active runs at an open PR's current head remain intact: GitHub cannot retry them while active, and retrying a downstream job would retain upstream placement. The watcher reports that limitation; completed runner refusals retain the supported failed-job retry. This change improves admission and detection but does not claim to relocate already queued active PR jobs.

Request: "Fix the class, not one job." Recovery constraint: "Only re-dispatches, no cancels of an open PR's run at its head."

Design: https://github.com/manaflow-ai/cmuxterm-hq/issues/5123#issuecomment-6515215902

Placement reads have a 90-second deadline that request error handlers cannot swallow. A timeout clears partial outputs and emits a default Mac placement; checkout, token minting, and watch upload are bounded within the job deadline. The observed #28460 placement had already completed its script and cleanup before GitHub reported a timeout, so this hardening does not claim to repair that runner acknowledgement failure.

When every admissible Mac target is excluded and no fleet client is enabled, the fast tier preserves the base owned queue and still runs. It neither skips required tests nor bypasses nightly/outage exclusions onto Blacksmith. #28391 adds client-only fleet execution for this deferred case after this PR lands.

Validation at 7d085f9774ae25f43b89b1954f7d10fb80203985: all 525 affected tests pass on the fleet (https://github.com/manaflow-ai/cmux/actions/runs/41114380935), and independent gpt-6-astra router review returned SHIP with no blocking findings. Expired Blacksmith outage/capacity data is tested independently of fresh owned gates, and placement carries the observation clock consistently through its admission checks.

Previously compared base warnings: release-notary's DigestTests.test_the_repository_highlights_validate and SigningHooksInStepWithGlaedaTests.test_every_listed_hook_is_one_glaeda_released_with_the_check failed identically on head run 41080623573 and base run 41071719011 (test_reds.py: own=false). The unrelated action-export-area job hit its five-minute timeout during checkout before running checks; a retry has been requested.

The routing, health adapter, and regression tests stay together because workflow placement and rescue share the admission contract. The fleet client is a separate execution mechanism in #28391.

Risk: admission depends on fleet health freshness; missing readings favor overflow once a queue has waited three minutes. Live-run recovery is limited by the no-cancellation constraint and Actions' retry API.

Agent session: agent session