improvement(execution): run sandbox workers on a dedicated worker host

Merged
WA
6d ago
+1362-73

Summary

  • The isolated-vm pool can open each worker as a WebSocket connection to a dedicated worker host instead of a local child process. The host (lib/execution/isolated-vm-host.cjs, shipped in the app image and run with node) spawns the unchanged isolated-vm-worker.cjs per connection and relays its IPC messages verbatim.
  • Remote mode turns on only where IVM_REMOTE_WORKER_URL + IVM_REMOTE_WORKER_TOKEN are set; with the URL unset nothing changes and no flag lookup happens. The AppConfig-only kill switch isolated-vm-local-workers (no env fallback) forces local workers for rollback, and an unreachable AppConfig keeps the host in use.
  • With a host configured, an unreachable host fails executions as a system error; it never silently falls back to a local worker. Flipping the kill switch retires workers on the old target after their in-flight executions finish. If a target change removes a selected worker before dispatch, the execution is re-queued.
  • fetch() and brokers stay in the app, so the host only relays frames. The client sends the settings the app owns in its first frame: limits (IVM_MAX_STDOUT_CHARS, IVM_MAX_FETCH_OPTIONS_JSON_CHARS), TZ, LANG and LC_ALL. Both modes therefore enforce the same limits and give the same Date/Intl results. The host validates each value's shape.
  • Client uses the standard global WebSocket (Bun and Node 22+); the host uses ws, now declared by apps/sim and copied into the image. The connection authenticates with a token in its first frame. The host sends a heartbeat frame so a client can detect a dead host, and announces draining on SIGTERM.
  • ffmpeg/ffprobe now run with a minimal environment (PATH, HOME, fontconfig vars) and -protocol_whitelist file,pipe before every input. Inputs are always temp files we write; the lavfi and concat inputs still work.

Type of Change

  • Improvement

Testing

  • isolated-vm.test.ts: two new tests. A configured host never falls back to a local worker when unreachable (fails with a fallback added). The kill switch forces local workers (fails if the flag is ignored).

  • ffmpeg.test.ts: new test that ffmpeg/ffprobe get no app env (fails without the env option); argv expectations updated

  • bun run test:isolated-vm-remote:e2e, run manually (not wired to CI): real host processes over real WebSockets under Bun. 17/17 checks pass: execution, worker parented by the host, limits come from the app, no host env in the worker, brokered fetch, a 5 MB result, timeout, cancel, auth rejection, an unparseable request target, protocol violations, capacity, crash → system error, drain, and fail-closed once the host is gone. Writes a JSON report to IVM_REMOTE_E2E_REPORT_PATH.

  • Parity corpus run in both modes, all 12 results identical (result, stdout, error name/message/line, termination): objects, undefined, console output, a thrown error, a syntax error, params, Date/Intl with TZ set only on the app, a timeout, brokered fetch, a 300k-char stdout truncation, a 2 MB result, async/await

  • Latency over loopback, two runs per mode (500 sequential executions with 20 KB params). The p50 deltas below are remote minus local, from run 1 and run 2.

    localremote
    cold start66 ms53–56 ms
    p501.28–1.38 ms1.45–1.69 ms (+0.07 / +0.41 ms)
    p992.5–2.6 ms2.6–3.6 ms
    500 concurrent458–483 ms512–513 ms
    40 CPU-bound concurrent117–130 ms119–123 ms
  • Checked locally that the whitelist keeps the lavfi silent-audio input and the concat-list input working, and rejects nested http inputs

  • biome, comment-hygiene, utils, source-conventions, test-patterns and file-names pass locally; type-check and the full suite run in CI

Checklist

  • Code follows project style guidelines
  • Self-reviewed my changes
  • Tests added/updated and passing (new tests pass the test-audit authoring gate)
  • No new warnings introduced
  • I confirm that I have read and agree to the terms outlined in the Contributor License Agreement (CLA)