Cloud dogfood: Run fan-out mode (N Runs at once, timed)

Merged
TE
17d ago
+328-1

Summary

Leo's goal for the agent lanes is to run them as Cloud Runs instead of local subagents. Nothing in the repo measured fan-out through the Run API, so this adds a run_fanout input (0, 1, 5, 20) to the existing Cloud VM dogfood journey workflow.

With run_fanout=N, the journey signs in its smoke account and creates N single-variant Runs at the same moment through POST /api/runs, instead of the TUI journey. Each Run's variant has a real instruction: codex reads README.md and replies with its first line. The receipt records, per Run:

  • admission time and HTTP status (a 429 at the team's 16-active-Run cap counts as rejected)
  • queue wait (Run created to attempt.started) and attempt time, from the Run's own events
  • the outcome and error, and the agent's last message read from the retained VM

The job summary prints the outcomes and p50/p90/max of each timing. Cleanup is the journey's: every VM of the smoke account and then the account are deleted.

  • web/scripts/cloud-vm/run-fanout.mjs: orchestration over injected api, now, sleep.
  • web/tests/cloud-vm-run-fanout.test.ts: specs pass the server's normalizeRunSpec, timings come from events, codex JSONL parsing, and a 5-lane run with 3 admitted (one fails) and 2 rejected at the cap.

Testing

  • Test commit first (module missing, red), then the module: bun test --isolate tests/cloud-vm-run-fanout.test.ts 5 pass on Huge Blue; tsc --noEmit reports no errors in the changed files.
  • Not run: the workflow itself. The cloud-vm-staging environment only deploys from protected branches, so the 5- and 20-Run runs happen after merge.

Risk

Low. Staging-only, opt-in workflow input whose default (0) leaves the journey unchanged. No product code changes.

🤖 Generated with Claude Code


View with [code]smith Autofix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need.


Summary by cubic

Adds a run_fanout input to the Cloud VM dogfood workflow so agent lanes can run through the Run API instead of the local TUI journey. It accepts 0–32 runs; the workflow exposes 0, 1, 5, and 20.

With run_fanout=N, the journey signs in with its smoke account and creates N single-variant Runs at once, each running a small codex task (read README.md, reply with its first line). The receipt per Run records admission time and HTTP status (a 429 at the active-VM cap counts as rejected, and the receipt names the cap), queue wait and attempt time from the Run's own events, and the outcome, error, and the agent's last retained message. Lanes still running at the deadline are cancelled so they can't create a VM after the account is deleted; run-time stats cover succeeded lanes only, and a cleanup error no longer rewrites an already-written receipt. The job summary prints outcomes plus p50/p90/max of each timing, and the journey exits non-zero if any lane fails.

Not yet exercised: the workflow itself — cloud-vm-staging only deploys from protected branches, so the 5- and 20-Run fan-outs run after merge. run_fanout defaults to 0, leaving the journey unchanged.

Summary by CodeRabbit

  • New Features

    • Added optional cloud VM fan-out mode for running up to 32 journeys concurrently.
    • Added workflow fan-out options of 0, 1, 5, or 20 runs.
    • Added reporting for outcomes, agent replies, elapsed time, queue activity, and run statistics.
    • Fan-out results include individual run outcomes, error details, and timing information.
    • Runs that exceed the configured timeout are cancelled automatically.
  • Bug Fixes

    • Fan-out execution reports failure when any requested run fails, while preserving the existing single-journey flow.