ci(nightly-next): G5 lets a commit whose nightly waits for its own tree promote before the tree publishes

Merged
TE
8d ago
+147-9

Leo: "it is unacceptable to have nightlies >30min out of date."

What is slow

A landing that changes the cmux-tui tree key waits for its tree before its NEXT nightly can start. Main's promote guard (G5) refuses a commit until its tree-commit marker exists. Measured on today's feat-cmux-next pushes (Actions jobs API, cmux-tui-artifacts runs 40913738554, 40918933039 and 40921878495):

step after the landingends at
the four build / build <target> legs (Linux, 32 vCPU)13-15 min
publish to R2 (commit-addressed, attested manifest)21-22 min
publish tree-addressed cmux-tui26 min
mark the commits whose tree is ready (the G5 marker)27-28 min

Only then does the nightly start, and the nightly itself takes 24-40 min from creation to publish-nightly (median about 30, runs 40868602625 to 40920248555). So a tree-changing landing publishes 52-58 min after it lands. About a quarter of landings change the key.

What changes

G5 (scripts/ci/promote-nightly-next.mjs) still refuses a 404 marker, with one exception:

  • the commit is a feat-cmux-next commit, not a train commit;
  • its own .github/workflows/nightly.yml, read through the contents API at that SHA, has the exact line NIGHTLY_NEXT_TREE_AT_INSTALL: v1.

That line is the contract a feat-cmux-next follow-up adds to nightly.yml. The nightly then:

  • compiles the app at once;
  • waits for the commit's own attested commit-addressed manifest before it installs the client, with the same install-cmux-tui-client.sh command and the same manifest, so the bytes are unchanged;
  • runs publish-nightly only once that commit's tree-commit marker exists and names the installed commit.

The check G5 makes moves into the run; it is not dropped.

Everything else refuses exactly as before:

  • a marker read failure (anything but a clean 404) never takes the exception, and the workflow is not read;
  • an unreadable nightly.yml refuses, and the reason names both the missing marker and the read failure;
  • so does a file without the exact line (a comment that mentions it, or a v2, does not count);
  • a train commit always needs its marker.

Until feat-cmux-next ships that nightly.yml, no commit has the line, so this changes nothing on its own.

Tests

tests/test_nightly_promotion_admission.py, 6 new cases (94 in all, all pass):

  • a commit with the line and a 404 marker promotes, with a notice;
  • the line must match exactly;
  • an unreadable nightly.yml refuses;
  • marker read failures never take the exception and read no workflow;
  • a published tree reads no workflow;
  • the reader asks the contents API for the raw file at the SHA, and refuses a directory.

The harness injects readWorkflow beside readTreeMarker. 3 of the 6 fail on base (the others assert behaviour base already has).

Performance

metricbeforeafterdeltaharness or run link
tree-changing landing to its nightly run created27-28 min (marker) + up to the 5-min windowabout 1-5 min, once the feat-cmux-next half landsabout -23 mincmux-tui-artifacts runs 40913738554, 40918933039, 40921878495 (jobs API); after: measured on the same harness from the first landings after both halves merge
this PR alone-no change (no commit has the line yet)0-

No new timer or loop. The only new API call is one contents read, and only for a commit whose marker is missing.

Risk: a promoted commit whose tree build then fails makes a nightly that fails at install and publishes nothing; the feed keeps its last build, as with a tip that does not compile. The feat-cmux-next half requests such a commit only while an artifacts run for its tree is active.

🤖 Generated with Claude Code

Agent session: agent session


View with [code]smith Autofix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need.


Summary by cubic

Shrinks the gap from a tree-changing feat-cmux-next landing to its nightly: G5 now lets a commit promote before its tree publishes, when that commit's own nightly is set up to wait for the tree at install.

  • promote-nightly-next.mjs still refuses a 404 marker, except for a feat-cmux-next commit whose .github/workflows/nightly.yml at that SHA contains the exact line NIGHTLY_NEXT_TREE_AT_INSTALL: v1.
  • The nightly compiles the app immediately, installs the client from the commit's attested manifest, and runs publish-nightly only once the commit's marker exists — the same G5 check, moved into the run.
  • The exception fires only on a structured clean 404, never on read failures or malformed markers; train commits, unreadable nightly.yml files, and files without the exact line refuse exactly as before.
  • Adds 6 test cases in tests/test_nightly_promotion_admission.py covering the exception path, the exact-match requirement, read failures, and the contents-API read.

Until the feat-cmux-next half ships that nightly.yml line, no commit matches, so this PR changes nothing on its own.

Summary by CodeRabbit

  • Bug Fixes
    • Improved nightly release promotion in a specific case where a commit’s tree marker is missing but its workflow confirms the required tree-at-install setting.
    • Promotion remains blocked when the workflow cannot be read, the required setting is absent or different, or the tree-marker check fails unexpectedly.
    • This helps keep eligible nightly releases moving while preserving safeguards against incomplete or unverifiable builds.