gh-merge-green: excuse a red lint check whose every finding the base also prints

Merged
TE
18d ago
+219-28

Summary

gh-merge-green refused every PR into feat-cmux-next while the base was red in any lint bundled into cmux-next checks (god files, concurrency, crash safety, l10n). Lint was never compared with the base, and --override can't waive it. #22013 is webview-only and was refused today over seven base reds: state roots, a blocking NSWorkspace call, 133 missing translations, motion, type scale and two key-input lists.

A red god-file, l10n or concurrency check now goes to scripts/ci/base_red_excuse.py like other red lanes, held to a stricter rule. Every line its failed steps printed must also appear on the base's latest run of the same job. Numbers, mktemp names and hashes are masked, and each step's script and env echo is skipped. A step that continues on error counts as failed when it logs ##[error]. One more missing translation or blocking call is still the PR's own and refuses. --override still refuses on any lint red, and risk evidence still has to be green.

Pairs with the feat-cmux-next PR that runs the checks job on pushes again. Until that lands, the base has no run to compare with, so these reds keep refusing, as today.

Testing

  • python3 -m unittest tests.test_base_red_excuse: 31 pass. New LintFindings cases cover findings all on the base, a finding the base lacks, a red step that printed nothing, a check green on the base, and temp names and hashes that differ between runs.
  • python3 -m unittest tests.test_gh_merge_green: 102 pass. test_red_lint_check_refuses_without_override now checks that the base is asked before refusing.
  • Red first: e2968a6d85 (tests only) failed 4 in test_base_red_excuse and 1 in test_gh_merge_green.
  • Real logs: printed() on the checks jobs of two different PRs' runs on the same base (runs 39315764306 and 39318045918) gives 212 lines each, and the sets are identical.

Observed / Not run

Observed: the runs above. Not run: a live merge through the new path. It needs a feat-cmux-next push run of the checks job, which the paired PR adds.

Review

A correctness review found three issues, all fixed in 0efc517da8 with red tests first (the test-only commit before it):

  • Masking every digit let a new onboarding.step2 key pass as the base's step1. Numbers are now masked only when no letter comes before them.
  • A set comparison let a second identical blocking call pass. Findings are now counted.
  • check-l10n.sh prints only its first 200 errors. A ... N more line now refuses.

On real logs, the checks jobs of two PRs on the same base still compare equal (228 lines each).

Known limit, the same as the generic base-red path: a step that stops at the base's red under bash -e never runs its later commands on either side, so a PR breaking one of those commands isn't seen until the base red is fixed.

Changelog

none

🤖 Generated with Claude Code

Summary by CodeRabbit

  • CI and Merge Checks
    • God-file, localization, and concurrency lint failures can be excused when the base run reports the same findings, including the same number of repetitions.
    • Unique or additional findings, incomplete output, and checks without comparable findings continue to block merging. Pending lint checks also remain blockers.
    • --override cannot bypass lint failures or risk-evidence failures. Differences in temporary paths and other run-specific details are accounted for when comparing findings.