ci: advisory fleet test-compile check for the test targets a change reaches
Summary
fleet/compile-check compiles only the app executable, which links no test target, and merge_compile_guard.py accepts its pass in place of a queued Release compile. Two PRs passed it today and then broke feat-cmux-next's test build: #23360 (a throwing call inside #expect without try, CmuxNextActionsTests) and #23163 (a FakeClient missing the requirement #21258 added, CmuxNextAppTests). On #23360 every Mac job that builds tests was still queued when it merged.
This adds an advisory check as a bridge, and leaves the merge gate alone:
fleet/compile-checkis unchanged: still the Linux app compile, same timing, same github-actions[bot] status.fleet-compile-check.ymlandmerge_compile_guard.pyare byte-identical to feat-cmux-next.- New
fleet-test-compile.ymlruns on pull requests into feat-cmux-next that touchPackages/orvendor/. A Linux job plans withscripts/ci/compile_test_plan.pyover the merge commit's diff: the CmuxNext test targets that depend on a changed target in the existing SwiftPM target graph (cmux_next_route.route, the swift test tier's routing), every one for a manifest or graph change, plus the macOS package lane's packages whose sources or path dependencies changed. An empty plan passes on that Linux runner in seconds and queues no Mac. A plan runspackage-test-lane.sh compile-tests BASEas a light fleet step, which builds each planned CmuxNext test target withswift build --target(no link, no run) and a lane package with--build-tests. The check,fleet/test-compile (advisory), goes red only when a planned test target fails to compile. The lane prints its failure marker as soon as a target fails, so a later package's setup failure can't hide it. Anything else (queue or run timeout, setup, the client) is a warning, and the job's 55-minute timeout outlasts the client's 20-minute queue plus 25-minute run timeouts, so a slow fleet never reads red. If SwiftPM refuses--target, the package falls back to one--build-testsin place of its remaining targets. - Nothing waits for it: the guard reads only the Release compile and
fleet/compile-check;gh-merge-greenis main-only and this workflow runs only on feat PRs.
It's a bridge until the Linux pool compiles test targets. The pool's darwin SDK can't find Testing today (job 1382fbdb06c0fc7c023643d3 at df5e2ebb1c03: no such module 'Testing'); cmuxterm-hq #2750 adds Xcode's platform search paths and reaches the pool with the next image bake. The per-target pool argument is manaflow-ai/cmuxterm-hq#2759.
Testing
Red/green on a fleet Mac with the build the new phase runs (the reviewed package-test-lane.sh suite phase, which runs swift build --build-tests over CmuxNext before one suite):
| head | result | fleet step |
|---|---|---|
#23360's head 4f867f5e57c2d943ac1f4e9743d42cc89730e683 | red, exit 1: PaletteMenuVocabularyTests.swift:74:106 and :79:101, "call can throw, but it is not marked with 'try'" | ed72e6e6f72d215befe9bb9b (aws-m4pro-8) |
feat-cmux-next df5e2ebb1c03db048843083530e86feeda2c1b2a | green: Build complete! (89.11s), then CmuxNextActionsTests (136 tests) passed | 54d316aebe3d339a9a2126eb (aws-m4pro-7) |
cmux-ci run --class light --script scripts/ci/package-test-lane.sh --ref <sha> --arg=suite --arg=Packages/macOS/CmuxNext --arg=CmuxNextActionsTests; cmux-ci log <step> on the tailnet prints each log. Each ran about 1.5 to 2 minutes after 11 minutes in the light-class queue, which is why this is advisory and off the merge gate.
tests/test_ci_compile_test_plan.py (linux-guard) was committed red first (ec4b51c696d0) and passes: #23360's diff selects CmuxNextActionsTests and its reverse dependents and nothing that doesn't reach CmuxNextActions; a new CmuxNextAppTests file selects only that target; the manifest selects every target; a CMUXAuthCore change selects it, its lane dependents and the CmuxNext targets that use it; iOS-only, docs, CI and app-host changes plan nothing; the phase refuses a bad merge base. The same file runs the compile-tests phase with a fake swift (committed red first in a9201f8aae30): a target that doesn't compile exits 1 with the marker; an earlier target failure stays red when a later package's setup fails; a setup failure alone has no marker; a refused --target falls back to --build-tests once per package; an empty plan builds nothing. It also runs the workflow step with a fake client to check how exits map to red, warning or green, and that the job timeout outlasts the client's. cmux-next.yml now runs it with the merge guard tests. tests/test_merge_compile_guard.py adds that a red or pending advisory check never holds a merge or stands in for the Release compile, and that the advisory workflow is not the fleet check the guard trusts. Passing on Linux (huge-blue): those two, test_ci_cmux_ci_client_guarded, test_ci_workflow_guards_are_wired, test_supplychain_security, test_runner_label_policy, test_ci_pr_runner_pool, test_ci_fork_runner_routing, test_ci_merge_queue_required_checks, test_ci_workflow_path_filter_parity, test_ci_workflow_run_sources, the reusable-workflow and cache guards, package-test-lane-suite-args.test.sh; actionlint; prepush-gate.py.
Observed / Not run
- Ran: the two fleet steps above, the pool probe (
no such module 'Testing'), the tests listed above,compile_test_plan.pyon #23360's diff (12 CmuxNext test targets includingCmuxNextActionsTestsandCmuxNextAppTests). - Not run: the
compile-testsphase andfleet-test-compile.ymlon the fleet. A changed allowlisted script runs on the fleet only after it lands (hqstep_reviewed.go), so the first real run is the first feat PR touchingPackages/after merge. Unverified there:swift build --target <test target>; if SwiftPM refuses it, the phase falls back to--build-testsfor that package. - Known: next_autoland holds a PR on any red check, including this advisory one, until teamleaderleo/leo-workspace#746 (skip checks named
... (advisory)) lands. Expected red here: this PR's own swift-package-tests fleet step is refused untilpackage-test-lane.shlands, as for every allowlisted-script change.
Changelog
none
Agent session: cosmic-gray
š¤ Generated with Claude Code