ci(base-red): read Swift Testing display-named reds as test names

Merged
AU
3d ago
+26-2

Summary

gh-merge-green still refused https://github.com/manaflow-ai/cmux/pull/35296 even though its app-host shard 2 fails the same three tests as main (CloudMachinesHeaderCountTests: head https://github.com/manaflow-ai/cmux/actions/runs/41848241642/job/126642789599, main https://github.com/manaflow-ai/cmux/actions/runs/41855478816/job/126683331325). The refusal was errors not on main run 41855478816: #-#-# #:#:#.#+….

Those tests are declared @Test("A capped plan …"), so Swift Testing prints ✘ Test "A capped plan …" recorded an issue, with no (). The Swift Testing pattern in both test_reds.FAILED and base_red_excuse.TESTS required …\), so the job had no failing tests. Head.mismatch then fell back to comparing error: lines, which here are WebKit ProcessAssertion … error: app logs whose hex pointers change on every run.

Both patterns now also accept a quoted display name. The quoted alternative is tried first, so a display name's recorded an issue at …: (x) failed can't stretch into the .+?\) match.

Testing

  • python3 -m unittest tests.test_test_reds tests.test_base_red_excuse: 152 OK. New tests cover display-named reds, including a parameterized one, in both modules, plus a shared-vs-new comparison.
  • Replay of the two real shard-2 logs: signature() now returns the same three test names for head and main, and test_reds.classify(head, main) is not own.

Follow-up to https://github.com/manaflow-ai/cmux/pull/35609.

🤖 Generated with Claude Code


Autofix with [code]smith
Open in [code]smith · Need help on this PR? Tag @codesmith-bot with what you need.


Summary by cubic

Fixes Swift Testing display-name red detection so gh-merge-green no longer refuses jobs whose only reds match the base run.

@Test("A capped plan …") prints ✘ Test "A capped plan …" recorded an issue with no (), so the existing Swift Testing patterns in test_reds.FAILED and base_red_excuse.TESTS didn't match fails from those tests. base_red_excuse then fell back to comparing app-log error: lines, whose hex pointers differ between runs, and refused to excuse a red that main has too. Both patterns now also accept quoted display names, and try the quoted alternative first so display-name issue details can't stretch into the old .+?\) match.

Summary by CodeRabbit

  • Bug Fixes
    • Swift Testing failures with quoted display names are now recognized alongside conventional Test …() names, including names with punctuation or an ellipsis.
    • Comparisons of test failures now account for quoted display names, helping identify when a failure is shared or newly introduced.
    • Existing failure-result phrases remain required when identifying these failures.