ci(rescue): recover nightly control runner stalls

Merged
TE
6d ago
+12-0

Problem

The nightly control job can be placed on the configured fleet Linux control runner, but the main-branch rescue did not classify that label as a recoverable nightly pool. A dead label could therefore leave decide queued until the run timed out.

Change

Treat non-hosted, non-Blacksmith labels on nightly decide as fleet control pools. The existing nightly rescue can then cancel and re-run a stalled attempt, whose second attempt uses the established fallback route.

Leo said: “For our nightlies, are they end to end on our fleet? We can’t be relying on Blacksmith that cuts out.”

Validation

  • cmux-ci test -- python3 -m pytest tests/test_ci_owned_pool_rescue.py

Changelog

  • Recover nightly control jobs queued on a fleet runner that stopped serving.

Agent session: agent session
🤖 Generated with Claude Code


View with [code]smith Autofix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need.


Summary by cubic

Fixes a gap in the nightly rescue that could leave decide jobs stuck queued when their configured fleet runner stopped serving. The rescue now treats any non-hosted, non-Blacksmith label on a nightly decide job as a fleet control pool, so the existing cancel-and-rerun logic applies and the rerun falls back to the established route.

Summary by CodeRabbit

  • Bug Fixes
    • CI pool recognition now includes jobs named decide that use nonstandard fleet-control labels, such as gcp-linux-control.
    • Standard labels beginning with blacksmith-, ubuntu-, macos-, or windows- remain excluded from this recognition.