#26398 ci(guards): every runner guard check runs, and a base-red check cannot excuse another

Leo (2026-10-27): "When I ask you guys to do things, I'm not just saying do this individual instance. I'm saying go do stuff."

This is the main-branch half of #26389, which does the same for feat-cmux-next.

Problem

tests/test_ci_self_hosted_guard.sh runs about 36 runner checks as one CI step under set -e. The first red check stopped all the checks after it. Also, gh-merge-green's base-red excuse (scripts/ci/base_red_excuse.py) compares a red step's error lines with the base's, and the guard printed no per-check error lines. So once one check was red on the base, a different check red on the change could be excused with it. On feat-cmux-next this is how a hosted runs-on: ubuntu-24.04 (chief-evals.yml) hid every later runner violation.

Change

• run_check wraps every check. Each one runs in a subshell, the script keeps going, and it fails once at the end with the list of failed checks. Each failed check, and each of its FAIL: findings, prints a plain error: line, so the base comparison is per check and per finding.

• check_build_lag_deriveddata_cache_path was defined but never called (dropped in 17c2498a3eb, then re-added without its call). It runs again, and it passes.

• New tests/test_ci_guard_check_isolation.py, wired into the preflight group and registered in test-execution.toml:

  • every check still runs after an earlier one fails;
  • every check_* function is called through run_check;
  • a fixture with two red checks, one of them also red on the base, gives the base comparison an error line the base lacks (not excused), while the same base red alone is excused.

  Against the old guard script, 6 of 6 tests fail.

Before: 35 of 36 checks ran (one never ran), stopping at the first red. After: 36 of 36 run, and all pass on main today, so no violations surfaced on main.

Risk: low. This is CI-only. A check that exits now ends only its own subshell.

Agent session: agent session

🤖 Generated with Claude Code

---

^{Need help on this PR? Tag @codesmith-bot with what you need.}

---

Summary by cubic

Makes the runner guard script run every check even when one fails, and stops a check already red on the base from excusing a different check red on the change.

• Each check now runs in its own subshell via run_check; the script fails once at the end listing every failed check, with a plain error: line per check and per FAIL: finding so base_red_excuse.py compares per check and per finding.
• Restores check_build_lag_deriveddata_cache_path, which was defined but never called (it passes).
• Adds tests/test_ci_guard_check_isolation.py to the preflight group, covering: all checks still run after one fails, every check_* goes through run_check, and a new red check beside a base-red one is not excused while the same base-red check alone is.

Before, 35 of 36 checks ran and the script stopped at the first red; now all 36 run and pass on main. CI-only, low risk.

Summary by CodeRabbit

• Tests
  • CI guard validation now checks that subsequent checks run even when an earlier check fails, and that failures are reported consistently.
  • Added coverage to verify that new failures can be distinguished from existing baseline failures.
  • Guard validation is included in the preflight and Linux guard test runs.