ci(next): a tip daemon suite gives the base a daemon-test verdict

Merged
TE
7d ago
+258-6

Problem

cmux-next daemon tests never runs for a feat-cmux-next push. Its if: skips same_tree_origin == 'push', and a push's own run is superseded before its tree publishes. So scripts/ci/test_reds.py / base_red_excuse.py have had no base log for that job since 04:34Z, and no pull request's daemon red can be compared with the base (#27692 is stuck on 17 red daemon tests, #27868 likewise). Base logs existed only from hand dispatches (41001517417, 41024114885).

Even a dispatch did not reliably help. Its tree wait marker (origin dispatch) was dropped as superseded whenever the tip moved before the tree published, which in a landing burst is every time: 41023525520 and 41022857098 ended with daemon tests skipped.

Change

This follows how tip_swift_suite gives swift-test a base verdict. The dispatcher side is #28261 on main.

  • cmux-next.yml takes a tip_daemon_suite input.
    • Path routing routes only the daemon tier, and only on refs/heads/feat-cmux-next.
    • The run is named cmux-next tip daemon suite · <tested sha>, using same_tree_sha in same-tree mode.
    • daemon-test's own if: is unchanged: a dispatch, and its same-tree run with origin dispatch, already run it.
  • Its tree wait marker carries tip_suite: daemon.
    • cmux_next_tree_notify.py treats such a marker as current after the tip moves, like the DEV daily's: it tests its recorded sha.
    • It passes tip_daemon_suite on, so the same-tree run keeps the name.
    • Only a daemon-only dispatch marker on feat-cmux-next heads may carry it. Anything else is rejected as malformed.
  • base_red_excuse.py looks up tip suite runs by name. If its window of the 100 newest base runs (up to 40 job reads) holds no verdict for cmux-next swift test or cmux-next daemon tests, it lists the base's workflow_dispatch runs and reads the newest tip suite runs of that job by name, at most 6 job lists. Base pushes crowd that window out within a couple of hours. test_reds.classify then judges per test as before. A crash or a cut-short run is still never a base red.

No workflow permission or secret changes.

Performance

Runner minutes added per base push (estimate). The daemon-test job took 9 to 13 minutes in 41024114885 and 41001517417. The base gets about 30 pushes an hour, and the dispatcher (#28261) starts at most 2 daemon suite runs an hour: when the daemon tier changed, or every 20 landings.

metricbeforeafterdeltaharness or run link
Mac runner-minutes per base push (daemon-test)0≤ 0.9 (≤ 2 runs/h × ~13 min ÷ ~30 pushes/h)+≤ 0.9runs 41024114885, 41001517417
REST reads per daemon/swift excuse lookup with no verdict in the window1 + 401 + 40 + 1 + ≤ 6+≤ 7 (only on that miss)tests/test_base_red_excuse.py

Tests

These pass on the branch:

  • tests/test_base_red_excuse.py:
    • a daemon red is excused by the tip daemon suite run past 45 base pushes with no verdict
    • a tip suite run green on the base still blocks
    • a DEV daily's jobs are never read
    • a job with no tip suite makes no extra lookup
  • tests/test_cmux_next_tree_notify.py:
    • a tip daemon suite marker dispatches its exact sha with tip_daemon_suite after the tip moved
    • malformed tip_suite markers are ignored
  • tests/test_cmux_next_tip_swift_suite.py:
    • the input, the run name, a daemon-only route and the marker field
  • tests/test_test_reds.py and tests/test_cmux_next_generated_fold.py also pass.

Stop condition: after this and #28261 land, the next base daemon-test result appears without a hand dispatch, and #27692's daemon reds are compared by name against it.

Agent session: agent session

🤖 Generated with Claude Code