ci(cmux-next): promote a smoked VM image snapshot to the dev channel from a workflow

Merged
TE
25d ago
+992-3

claude-sr2's decision, word for word: "Same secret, same environment, contents: read, no new tokens: that's no new trust, and the dev channel is ours. Build the promote workflow exactly as you described, including the guarded delete_snapshot input."

What it changes

  • .github/workflows/cloud-vm-image-promote.yml (new, workflow_dispatch only). Inputs are channel (a choice with dev as the only option, and refused again in the first step), snapshot (sh-…), bake_run and an optional delete_snapshot. It runs in environment cmux-next-vm-image-dev and refuses any ref other than feat-cmux-next. Steps:

    1. Check the inputs: the id formats, bake_run is required with snapshot, and delete_snapshot may not be the snapshot being promoted.
    2. Check that bake_run is a successful feat-cmux-next workflow_dispatch run of cloud-vm-image-bake.yml in this repo.
    3. Download that run's cmux-vm-image-bake-<run> artifact and append its channel-dev.json, minus schema and env, to channels/dev.json history (web/scripts/cmux-vm-image/promote-record.ts). It refuses a candidate whose id, run, smoke result, name prefix or var doesn't match, and it never edits an existing entry.
    4. Run bun ../images/cmux-vm/promote.ts --channel dev --snapshot <id> --var <candidate's var>. That is promote.ts's own fresh-clone smoke on the dev account.
    5. Upload git diff of images/cmux-vm/channels/dev.json and backend/apps/api/wrangler.jsonc as the artifact promote-dev-<snapshot>.diff (kept 3 days).
    6. Write the diff, the fresh-clone smoke record and promote.ts's rollback lines to the job summary.

    It pushes nothing and opens no PR. A lane applies the diff on a branch, opens the channel PR, and lands it after review; the next development deploy ships it.

  • delete_snapshot goes through a new cleanup.ts --snapshot <sh-id> mode, which calls freestyle's vms.snapshots.get/delete, the same client the ledger cleanup already uses. It refuses:

    • an id that any images/cmux-vm/channels/*.json string mentions: pointer, previous, team_vm, history, prose notes;
    • an id that backend/apps/api/wrangler.jsonc mentions;
    • a snapshot whose slug or display name is mentioned in any of those files;
    • a snapshot not named cmuxnp-dev-vmimg-*.

    The scan fails closed and catches hidden mentions:

    • a channel file that doesn't parse refuses the delete;
    • the scan covers parsed values and keys, every JSON-decoded string literal (so a duplicate key or an escape can't hide an id) and raw lines;
    • matching is case-insensitive and on whole tokens.

    It checks both the job's checkout and feat-cmux-next fetched just before the delete, so a channel PR that lands mid-run still protects its snapshot. It prints every reference it finds (<file>: <json path> or <file>:<line> <key>). If the job also promoted, the check reads the promoted files, so the old pointer, now previous, stays protected.

  • Tests. web/tests/vm-image-cmux-vm-promote-workflow.test.ts (wired into cloud-vm-image-lock.yml) checks:

    • dispatch only, no pull_request_target;
    • permissions exactly contents: read and actions: read, and no job-level permissions;
    • environment cmux-next-vm-image-dev;
    • cancel-in-progress: false;
    • no expression interpolated into a run script;
    • the input refusals;
    • key handling: it runs the real step scripts with a stub bun and checks that the key is not in argv or the environment, the file is mode 600, and the file is gone afterwards;
    • every reference refusal, plus the real channel files protecting the serving image and all history entries;
    • promote-record's refusals.

    tests/test_supplychain_security.py pins the same permissions, environment and trigger.

  • Docs. "Promote from CI" in plans/cmux-next/release-rails.md.

Permissions and key

  • Workflow permissions are contents: read and actions: read, the latter only so actions/download-artifact can read the bake run's artifact. There is no write scope, no PR token and no pull_request_target, and checkout runs with persist-credentials: false.
  • The key is the existing FREESTYLE_API_KEY secret of the cmux-next-vm-image-dev environment, the same secret and environment cloud-vm-image-bake.yml uses.
  • Each step that uses the key first copies it to an unexported shell variable and unsets the env var, so no child process (mktemp, git, grep, bun) inherits it. It then writes the key to a mktemp file in $RUNNER_TEMP under umask 077/chmod 600. promote.ts and smoke.ts read it through FREESTYLE_API_KEY_FILE, and a trap deletes the file on exit. The key is never on a command line or echoed.
  • The diff step refuses to upload an artifact that contains the key.

Risk

  • Low. The workflow can only run when someone dispatches it, and it changes nothing in the repo or on any Worker.
  • The one provider write that doesn't come from promote.ts is delete_snapshot, which refuses any snapshot a channel or the Worker config names.
  • No existing workflow changes behavior: cloud-vm-image-lock.yml only runs one more test file, and cleanup.ts --ledger is unchanged.

Testing

  • cmux-ci test (fleet Linux) at the head: bun test tests/vm-image-cmux-vm-promote-workflow.test.ts tests/vm-image-cmux-vm-bake-workflow.test.ts passed 58 tests, and actionlint is clean on both workflows.
  • The key test shims every external command the key steps run and fails if any of them sees the key.
  • tests/test_supplychain_security.py passed 7.
  • The other workflow guard tests fail there only on files this PR doesn't touch (chief-evals.yml, cmux-next.yml, nightly.yml, ci.yml), so those are base reds.

Size: 992 changed lines, 452 of them the test file. That is over the ~800 guide. I kept it as one PR because the workflow, its delete guard and their tests went through six review rounds together.

First use: sh-305c4f86261349909f5b0330781b7623 from bake run 40270232246 (source 41c5481957f3, smoke PASSED, 25 checks).

Agent session: agent session

šŸ¤– Generated with Claude Code