ci: bound cmux-tui tree-preflight's fallback owner election

Merged
TE
9d ago
+255-8

When the tree-claim authority can't be asked, cmux-tui tree-preflight falls back to electing the key's owner from the run list. That election fetched each active feat-cmux-next run's commit and computed its tree key with no time bound, so a slow election ran into the job's 6 minute timeout. The election now ends at a deadline, with this run building, instead of in a killed job.

Run 40650195591 (commit a09b77c1) shows it: the authority answered HTTP 000 at 07:28:04, and the election ran until the job timeout killed it at 07:33:39. GitHub reports that as cancelled and nothing re-runs it, so the commit never built its tree and nightly-next couldn't promote it.

What changed in the inspect step:

  • The election ends 200 s into the step, measured as the step's own bash SECONDS (election_deadline; ELECTION_DEADLINE_SECONDS overrides it in tests). Slow tree checks or a slow claim before the election shorten it instead of pushing it past the job timeout.
  • The run-list request gets what's left of the deadline (at most 15 s) and is skipped once the deadline has passed. Each candidate's git fetch and tree key read get what's left (at most 30 s) under timeout --kill-after=2, so even a call that ignores SIGTERM ends. Neither starts with less than a second left.
  • When the deadline is reached, it warns (::warning::the owner election stopped at its deadline, ...) and keeps the lowest owner found so far. With none, this run owns the key and builds. A duplicate build is acceptable; a lost build isn't.
  • The newer-head fetch earlier in the step had no bound either. It now gets timeout --kill-after=2 30, and it already fails open.
  • The same-SHA election, cancel-in-progress and the concurrency group are unchanged.

The bounds outside the election already existed and this PR doesn't change them: the tree completeness curls, the claim authority's ~50 s, and the 15 s same-SHA lookup.

Testing

python3 tests/test_cmux_tui_election_budget.py runs the inspect step with a fake curl, git, claim script and tree key script:

  • Base (cfd8a475252): fails 10 checks. A hanging fetch or key read runs until the harness kills it at 40 s, a slow election examines every candidate, and an election past its deadline still reads the run list and fetches.
  • Head: passes. A quick election still elects the lowest active run of the key. A slow one stops at a 3 s deadline and keeps the lowest owner it examined. A hanging fetch, a hanging key read, and a fetch that ignores SIGTERM each end in 3.5 to 5.5 s, with this run building (owner_run_id empty, run_macos=true). An election that starts past its deadline reads nothing.

The test is registered as lane = "manual" with its reason, and it's listed in UNWIRED in tests/test_ci_workflow_guards_are_wired.py, like the two existing entries there. The repo adds no new CI tests (CLAUDE.md, 2026-10-23), so run it by hand when this step changes.

Observed / Not run

  • Ran the new test (red on base, green on head), plus tests/test_cmux_tui_pin_duplicate_skip.py, test_cmux_tui_lane_daemon_tests.py, test_cmux_tui_superseded_push.py and test_cmux_tui_owner_wait_defers.py, which run the same job's scripts; all pass. actionlint reports only an SC2034 warning at line 1535 that's also on base. python3 -m unittest tests.test_cmux_next_checks_workflow (160 tests, including PushPreflightBehavior, which runs this step), tests/test_ci_workflow_guards_are_wired.py, validate_test_execution_registry.py --pr-lanes feat-cmux-next and scripts/cmux-next/prepush-gate.py all pass.
  • Not run: a real election against GitHub with the authority down.

Risk

Low. Only the fallback path changes, and only when the claim authority is unreachable. The worst case is that this run builds a key another active run is also building: one extra tree build, and publish keeps the first write-once publication. Today's failure mode loses the tree.

Changelog

none

Agent session: agent session

šŸ¤– Generated with Claude Code