ci(nightly): remote-daemon build and attest jobs take the Linux runner set (base red)

LA
7d ago
+6-3

Summary

Base red (priority zero). On feat-cmux-next, tests/test_nightly_trusted_routing.py
test_every_linux_nightly_job_takes_a_runner_set_member fails with 20 failures, for two jobs.
build-nightly-remote-daemon (nightly.yml:3077) and attest-nightly-remote-daemon (nightly.yml:3129) resolve to
blacksmith-4vcpu-ubuntu-2404, which is not in the trusted runner set
{blacksmith-16vcpu-ubuntu-2404, blacksmith-32vcpu-ubuntu-2404}. This makes nightly-guards / release-notary red on every feat PR (for example #28114).

Cause: #25935 (85628c020a3, "free Mac build slots from portable nightly work") moved both jobs off the Mac builder onto the 4 vCPU label.

Fix: both jobs use the runner-set expression of the other Linux nightly jobs. That expression gives 16 vCPU when vars.LINUX_RUNNER names it, else 32. They stay off the Macs, which was the goal of #25935. No new job, step or gate.

Two rules that conflict: why one assertion changes

@teamleaderleo, both rules are yours, and they cannot both hold for attest-nightly-remote-daemon:

  • #27125 (9127e619f76), test_nightly_trusted_routing.py: "Never LINUX_RUNNER as-is: 16 vCPU when it names 16, else 32, whatever it holds (empty, the overflow switch's hosted value, an old 4 vCPU value, 16 or 32)." This is the trusted-routing rule for nightly jobs that produce release artifacts.
  • #25935 (85628c020a3), test_gh_budget_wait.py: self.assertIn("blacksmith-4vcpu-ubuntu-2404", job["runs-on"]), with the job comment "a quota wait must neither occupy a Mac nor delay the feed."

The lead decided that the security rule wins over the cost rule. So only that one assertion changes: it now checks that the job holds no Mac (assertNotIn("macos", ...)) and takes the runner set. The other assertions of the test are unchanged.

Cost: the attest job can wait up to 65 minutes for gh quota, and it now waits on a 16 or 32 vCPU runner, not on 4 vCPU. Please choose a cheaper design if you want one, for example a small trusted runner class added to the set (and to test_nightly_trusted_routing.py) for quota waits.

Testing

  • Red, feat tip 97133a48ad0: python3 -I tests/test_nightly_trusted_routing.py gives FAILED (failures=20), all 20 in build-nightly-remote-daemon and attest-nightly-remote-daemon.
  • Green, head b87f47ae211: the same command gives OK.
  • python3 -I tests/test_gh_budget_wait.py: OK on the tip and on the head.
  • python3 -I tests/test_ci_blacksmith_linux_failover.py: OK on the head.

Observed / Not run

  • Observed: the Python workflow tests above, run on this VM. They read the workflow file only; no build.
  • Not run: a nightly run on the new runners (this changes only the runner label of two existing jobs).

Changelog

none