base red: tooling-tests registry, cmux-next checks and release-notary (highlights, capabilities, step coverage, arm64 recorder)

Merged
LA
5d ago
+181-13

Fixes the checks that are red on every feat-cmux-next pull request, whatever it changes (seen on #30407, #30496, #30501, #30503, #30509, #30518). Each commit fixes one check and names the change that broke it.

CheckFailing partCauseFix
tooling-testsvalidate_test_execution_registry.py: "tests/test_ci_blacksmith_linux_sizes.py: linux-guard lane is not run by any workflow"#30360 registered the test as linux-guard and added no ci-guards.yml steplane = "manual" with how to run it (no new CI tests, Lawrence 2026-10-23). The six unregistered tests in the same log get entries too: test_nightly_dedicated_mini.py is linux-guard (ci-guards.yml already runs it); the other five are manual
cmux-next checks / Checks job structuretest_cmux_next_checks_workflow.py: 14 errors KeyError: 'run'#30221 added two steps after tree-preflight's inspect step, and the test read the job's last stepThe test reads the step by id (head, inspect). Assertions do not change
cmux-next checks / Tier routingtest_cmux_next_route.py: AgentPageHibernationTests.swift flagged as running page JavaScript on a built bundle#30312 added waitUntilLoaded to that suite. Its pane loads AgentPaneSource.devURLVariable (a loopback port nobody serves), not the bundleThe check counts AgentPaneSource.devURLVariable as the suite's own document, like loadHTMLString
cmux-next checks / Daemon capabilities export is fresh, and Script tests (daemon-capabilities-fresh.test.sh)Export does not list agent-session-start-no-cwd-v1#30363 added the capability and did not update the exportRegenerated with check-daemon-capabilities-fresh.py --write (the renderer base-autofix uses)
cmux-next checks / String tablesstore.updates.available: ja, km, ko, th, vi, zh-Hans, zh-Hant "expected a plain stringUnit or English's plural forms"#30288 gave those languages only the other formAdds their one form with the other text, as home.stopAll.couldNotStop does. Also adds the 12 languages #27883 left out of action.sidebar.compactList.toggle (needs_review)
nightly-guards / release-notaryscripts/whats-new/test_digest.py: three highlights "need a tryIt action/deeplink or a docs link"#29920, #30334 and #27905 added highlights with noneEach links its PR (pr:), as #29939 did
nightly-guards / release-notarytests/test_cmux_next_static_capability_runtime_only.py: agent-session-start-no-cwd-v1 missing from runtime_only#30363's capability is pushed by identify_capabilities at runtime, so the static manifest never lists it (every nightly's --static-only check would refuse it)It joins runtime_only in check-daemon-capabilities.sh, as #30447 did for agent-session-settings-v1
nightly-guards / release-notarytests/test_nightly_step_coverage.py: a stale and a missing entry#30200 removed "Read the visual gate's verdicts" and renamed "Fail with the gate when it holds the build" to "Record failed advisory verdict".github/nightly-step-coverage.json follows: stale entry removed, renamed step keeps its plumbing entry
nightly-guards / release-notarytests/test_nightly_next_arm64_only.py: CalledProcessError in "Build nightly app (Release)"#30107 made that step source scripts/cmux-next/lib/compilation-cache.sh; the test's scratch tree did not have itThe scratch tree gets the real file, as it does scripts/lib/rust-targets.sh. Assertions are the same
nightly-guards / release-notarytests/test_nightly_next_identity.py: test_each_app_reads_the_feed_its_publisher_writes#30278 made NIGHTLY_FEEDS a YAML anchor and aliases; the test read the raw textThe test's job_env resolves an alias to its one anchor and drops the anchor name, as YAML does. Asserted values are the same

Companion workflows: its red on #30407 (test_car_guards.py) is already green at the tip. No check was made weaker, and no CI job, step or gate was added.

Red and green

Each command was run on this Linux VM. "Red" is at base 03423dead871; "green" is at this head (after merging origin/feat-cmux-next):

  • python3 scripts/ci/validate_test_execution_registry.py --pr-lanes feat-cmux-next --base-sha HEAD^1: red exit 1, green exit 0 (594 tests).
  • python3 tests/test_cmux_next_checks_workflow.py: red FAILED (errors=14), green 165 OK.
  • python3 tests/test_cmux_next_route.py: red FAILED (failures=1), green 112 OK.
  • python3 scripts/cmux-next/check-daemon-capabilities-fresh.py: red exit 1, green exit 0. bash scripts/cmux-next/tests/daemon-capabilities-fresh.test.sh: green ok.
  • scripts/cmux-next/check-l10n.sh .: red 7 errors (19 without L10N_STYLE_WARN_MAX), green 0 errors.
  • python3 scripts/whats-new/test_digest.py: red FAILED (failures=1), green OK.
  • python3 tests/test_cmux_next_static_capability_runtime_only.py: red FAILED (failures=1), green OK.
  • python3 tests/test_nightly_step_coverage.py: red FAILED (failures=1), green OK.
  • GITHUB_STEP_SUMMARY=/tmp/s python3 tests/test_nightly_next_arm64_only.py: red CalledProcessError, green PASS.
  • python3 tests/test_nightly_next_identity.py: red 7/8, green 8/8.
  • Every script in the Checks job structure, Tier routing and Companion workflows steps passes at this head, except tests/test_next_batch.py. That test fails here only because this VM has git 2.43 and it needs 2.46 for attr.tree; CI is green on it.

Observed / Not run

  • Observed: each command above, red at base and green at head. test_pr_lane_coverage.py, test_ci_test_execution_registry.py and prepush-gate.py pass.
  • Not run: no Swift build or test (none of the changes is Swift source).

Still red in release-notary (not fixed here; each needs its owner's decision, comments are on the culprit PRs)

  • tests/test_nightly_ghostty_helper_cache.py and tests/test_nightly_workflow_size.py (nightly.yml is 508,735 bytes, over the 508,000 headroom): #30284.
  • tests/test_nightly_universal_build.sh (pinned helper runs-on) and tests/test_ci_warm_sources.py (10 nightly.yml jobs that can now take a Mac have no warm-sources class): #30249.

Changelog

none