test(cmux-next): a test that stops the main run loop no longer ends the run

Merged
TE
7d ago
+334-25

The cmux-next swift test job reported a fraction of its tests and still exited 0. This is the cause.

"Find the test or harness path that returns from async main early, or calls exit, before XCTest finishes."

What was happening

2,722 tests started and never finished. No crash, no signal, no failing test — the process left async main normally, so the job read as a pass:

main + 132
  swift_task_asyncMainDrainQueue
    CFRunLoopRun          <- the outermost run of the main run loop
  exit(0)
    __cxa_finalize_ranges

A test drives AppKit's event queue. HomeOpenFocusesComposerTests' click harness posts events with NSApp.postEvent (:55) and pumps them with NSApp.nextEvent (:44). AppKit answers that by calling CFRunLoopStop on the run loop it finds running — and the outermost run of the main run loop is swift_task_asyncMainDrainQueue's own CFRunLoopRun, the one driving the entire Swift Testing run. Stopping it returns from the drain queue, main + 132 falls through into exit(0), and every test that had not yet reported is simply gone.

Nothing in the harness is doing anything unusual. Using NSApp.nextEvent is how you pump events in an app; the problem is that under swift test the run loop it stops is the test run's own.

The fix

A test must not be able to end the run by using AppKit the way an app does. CmuxNextTestRunLoopGuard installs a run-loop observer that keeps the outermost run alive: when the drain queue's CFRunLoopRun is stopped from underneath it, the guard re-enters it. AppKit's stop then ends the nested run the harness is pumping, and nothing else.

The guard does not try to tell a legitimate stop from a stray one: it re-enters on every return, and its caps only limit repeated or spinning returns (it stands aside and says so in its trace). A finished run still ends normally because swift-testing ends the process by calling exit() from its own worker job, which works at any run-loop depth.

TestProcessExitTests covers the exit path itself: an exit() from async main now prints its stack instead of ending the run silently, so the next instance of this names itself rather than taking another investigation.

Why not fix the test instead

Making this one harness avoid NSApp.nextEvent would fix this instance and leave the hazard: any future test that pumps AppKit events the same way silently truncates the run again, and the symptom is a green job. The guard makes the whole class impossible.

Measured

On the suite, with the guard in place: 4 re-entries, 0 "standing aside", and a complete summary line — the run loop was stopped four times by the harness and the test run survived all four.

This PR adds zero files under Sources/

All 6 files are Tests/, Tests/Support/ or Package.swift. The guard is linked into the test target and is not in the app:

file
Tests/Support/CmuxNextTestExitTrace/CmuxNextTestRunLoopGuard.cthe guard
Tests/Support/CmuxNextTestExitTrace/CmuxNextTestExitTrace.c, include/…hthe exit trace
Tests/CmuxNextAppTests/TestProcessExitTests.swiftcovers the exit path
Tests/CmuxNextAppTests/HomeOpenFocusesComposerTests.swiftthe harness that found it
Package.swiftlinks CoreFoundation into the test-support target

Changelog

none

No capture: test-support only; no app surface and nothing to show.

Scope

Split out of #25391 under the ~800-line rule, as the cause half. The other two are separate PRs:

  • #25856 — the gate that makes a truncated run fail instead of reading as a pass
  • #25857 — the crash-report diagnosis (SWIFT_BACKTRACE, both report directories)

They are independent of this one and of each other, and each is reviewable on its own. The complete-run claim belongs to a job carrying this PR and #25856 together: this one stops the truncation, that one makes any remaining truncation visible.

Note for review: the guard test that #24424 added for this area is vacuous. CFRunLoopRunInMode on a private mode with no sources, timers or observers returns kCFRunLoopRunFinished immediately, so its #expect(result == .finished) is satisfied by emptiness rather than by the stop being absorbed. It would pass with no guard at all.

Observed / Not run

Observed: the symbolicated main + 132 → exit(0) stack above, from the real job; the guard's 4 re-entries and complete summary on the suite; prepush-gate.py --base --head reporting no new violations at this exact commit.

Not run: the full CmuxNextApp suite at this head reporting every one of its tests. A fleet run of scripts/ci/run-swift-testing-suites.sh at this commit is in flight and I will post its result here; until it reports, the claim made above is the 4 re-entries and the summary line, not a whole-suite green.

🤖 Generated with Claude Code

Agent session: agent session

ActionRunTimingTests.onlyAPersonDrivenNestedLoopCountsAsDwell (ActionRunTimingTests.swift:288) is a base flake, not this PR: run alone 5 times on the same fleet worker, it failed 2/5 with the guard (c0850549d1) and 1/5 without it (base 3ddddb8cbe), with the same assertion (watch.wakes → 0) and the same ~3.0-3.1 s passing runs.

Swift-test reds that runs used to hide by truncating (21 at 51578eea8c, including WindowZoom, the stale exports and the AcpmuxUnresponsiveWatch, layout, hover and style tests) were triaged as base reds and assigned to the swift-test triage helper; none blocks this PR.

Review (codex gpt-6-astra xhigh): SHIP at 95e2e1e3d36c941319798ce7e8c3fb1109f8f6e9

Claimed by incremental-build on 2026-10-27 after the owning lane was unresponsive >10 min (pr-owner claim).

Agent session: agent session


View with [code]smith Autofix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need.