#32498 ci(nightly): the Mac signer jobs read the signed app and final DMG in parallel ranges

Why

The NEXT nightly's path from landing to feed should fit in 30 minutes and takes about 45. Two of the losses are actions/download-artifact on the nightly's Mac runners. It reads the artifact's blob in one stream with no stall timeout.

• signer-dmg-image, "Download the signed app" (355 MB): in the last 26 completed nightly-next runs the first try took a p50 of 133 s and a p90 of 300 s. It hit its 5-minute limit in 4 runs. In run 41478567483 both tries ran out (16:43:41 to 16:53:41, with no bytes reported after "Starting download"). The macOS signer covered the build about 11 minutes later, and by then a newer build had published, so publish-nightly skipped it as superseded. That tip never reached the feed. The same artifact downloads in a p50 of 9 s (max 98 s, n=51) in signer-app-notary and signer-dmg on hosted Linux.
• verify-final-dmg-nightly, "Download final DMG" (257 MB): p50 89 s, p90 274 s, max 446 s. 6 of 26 runs took 200 s or more. In run 41481821056 this step took 446 s of the job's 537 s; it is the 9-minute verify. The other steps were normal: verify 33 s, Apple's checks 24 s.

From a Linux VM, the same 257 MB blob (its signed URL taken through the REST redirect, which points at the same blob host) took 30.0 s in one stream, 4.6 s in 8 ranges and 2.9 s in 16 ranges, with the digest matching every time. The limit is per connection.

What

• New scripts/ci/fetch_run_artifact.py (stdlib only):
  • Asks the artifact service that download-artifact itself uses (twirp ListArtifacts and GetSignedArtifactURL at ACTIONS_RESULTS_URL) with the job's runtime token. It reads the run and job backend ids from the token's Actions.Results scope. No GITHUB_TOKEN permission is needed, and no job's permissions change.
  • A run step does not see the runtime token, so a pinned actions/github-script step (v9.0.0, the pin used elsewhere in the repo) starts the script and hands it its own environment. Nothing is written to $GITHUB_ENV except the SIGNER_APP_FETCHED flag. The token goes to the artifact service only, never to the blob host.
  • Reads the signed blob URL in 16 parallel byte ranges (at least 16 MiB each). It is the same Azure blob SAS host download-artifact reads from (*.blob.core.windows.net), and that host answers Range requests with 206.
  • If a range sends nothing for 20 s, it asks again from the byte where it stopped, with a fresh signed URL. Each range gets up to 5 tries, and the whole download has a 240 s budget.
  • Checks the zip against the SHA-256 digest the service recorded at upload, then unpacks it into the same directory download-artifact uses, refusing members that would land outside it.
  • On any failure (no runtime token included) it exits non-zero and leaves the directory empty.
• signer-dmg-image: the ranged step runs first (continue-on-error, 5 min). The existing download-artifact step and its retry run only when the ranged step did not set SIGNER_APP_FETCHED. It is an env flag rather than a step outcome because signer-dmg reuses those two anchored steps, has no ranged step, and so keeps its current route.
• verify-final-dmg-nightly: the same pattern. "Download final DMG" (download-artifact) runs only when steps.fetch-final-dmg did not succeed.
• Can the final-DMG download be dropped? No. The final DMG is signed, notarized and stapled on Linux by signer-dmg, and verify has to read those exact bytes on a Mac. The image is already smoked in parallel (launch-smoke-nightly, smoked_in_job), so the transfer is the only part left to cut.

Every gate stays: both notarizations, the static verify, Apple's checks, and the launch smoke that publish needs. The digest checks downstream (APP_SHA256 on signed-app.tar, the tested DMG digest) are unchanged.

Performance

metric	before	after (expected)	delta	run link
signer-dmg-image signed-app download, p50	133 s	~25 s	~-108 s	26 runs from 41448288706 to 41485637444
signer-dmg-image signed-app download incl. retry, p90	384 s	~60 s	~-324 s	same 26 runs
signer-dmg-image runs that hit the 5 min limit	4 / 26 (1 lost its tip)	~0 (stalled ranges resume)	-4	41478567483
verify-final-dmg-nightly final-DMG download, p50	89 s	~20 s	~-69 s	same 26 runs
verify-final-dmg-nightly final-DMG download, max	446 s	~60 s	~-386 s	41481821056
same 257 MB blob from a Linux VM: 1 stream / 16 ranges	30.0 s	2.9 s	-27.1 s	measured locally against artifact 13164292369

The "after" column is an estimate: about 6 times the single-stream rate, from the Linux measurement. The first nightly-next runs after this lands will give the real numbers, which the ranged step prints (MB, seconds, MB/s, range retries). Blacksmith minutes are unchanged: both jobs run on owned Macs.

Tests

• python3 tests/test_fetch_run_artifact.py (new, added to ci-guards' "Validate the release signer scripts"). Against a fake API and blob host it covers: a ranged read and unpack; a stalled range resuming with a fresh URL; a digest mismatch leaving nothing behind; a range that never finishes failing within the budget; and a zip member that escapes the destination being refused.
• python3 tests/test_nightly_signer_jobs.py (new wiring test: ranged step first, fallback conditions, permissions, signer-dmg unchanged), tests/test_nightly_step_coverage.py (two new coverage entries), tests/test_nightly_sign_runner.py, tests/test_nightly_next_launch_smoke.py, tests/test_nightly_trusted_routing.py, tests/test_nightly_dedicated_mini.py, tests/test_nightly_next_post_publish.py, tests/test_ci_self_hosted_guard.sh, tests/test_ci_upload_artifact_retention.py: all pass.
• The pinned actionlint (1.7.12-manaflow.1) passes on nightly.yml and ci-guards.yml.
• tests/test_supplychain_security.py fails the same way on base (sentry-dsyms-nightly), so it is not this PR's red.

Agent session: agent session

🤖 Generated with Claude Code