feat(automations): run automations in cloud workspaces

Merged
SA
1mo ago
+14129-773

Summary

Automations can now run in the cloud, in three ways:

  1. New cloud workspace per run. Each run starts a workspace from an environment, and the agent starts on first boot.
  2. Pinned cloud workspace. Each run wakes that workspace and starts the agent in it.
  3. Continue the last session. The prompt goes into the agent session the previous run left in the pinned workspace.

This works from the desktop, the CLI (--cloud, --environment, --workspace) and MCP.

Also includes:

  • One run lifecycle shared by host and cloud dispatch.
  • The target rules for automation.create and automation.update moved into a pure, tested planner. automation.ts is now smaller than on main.
  • Desktop: the automation sentence reads device first ("on [device] in [project/environment] using [workspace] …"), and the "running …" chip appears only when a workspace is pinned. Both apply to machine and cloud automations.
  • The db-migrations skill now asks for a rehearsal under load only for hot or large tables.

Screenshots and end-to-end results:

Why

Before this, picking "Cloud" in the automation editor failed on save ("Host cloud is not registered"). Dispatch only reached machines through the relay, and cloud workspaces are deliberately not on it.

How it works

Data model: automations.target_host_id = "cloud" (CLOUD_HOST_ID, now in @superset/shared/host-routing), plus:

  • automations.cloud_workspace_id: the pin, with a foreign key to cloud_workspaces, set to null if that row is deleted.
  • automations.environment_id: the environment new workspaces start from, with a foreign key to environments.
  • automation_runs.cloud_workspace_id: the run's workspace.

v2_workspace_id stays a host workspace only.

Dispatch (automation/dispatch.ts, cloudDispatch.ts, run.ts):

  • One dispatchRun inserts the run and always settles it, so no path leaves a run in dispatching.
  • Pinned box: wake it (wakeCloudWorkspace), list its agents (see Known limitations), then POST agents.run straight to the box's host-service with its host secret.
  • No pin: startCloudWorkspace with launch, which is the same code path cloudWorkspace.create now uses.
  • Pinned box gone (deleted, failed, can't resume, or stuck provisioning for more than 10 min): start a new box and re-pin to it.
  • Before each run: the owner must still be a member of the org and still have cloud access.
  • Deadline: getting ready (access checks, waking the box, listing its agents) gets 30 s. Past it, the run is dispatch_failed with the code cloud_not_ready. Once agents.run is sent or a new box is started, it runs to completion under its own timeout.
  • The automation dispatch route's maxDuration goes from 60 s to 120 s, so a slow new-box start, or a host workspace create with its 90 s relay timeout, isn't cut off.
  • New error codes: cloud_not_ready, cloud_access_denied and cloud_environment_unusable, each with desktop guidance.
  • Pull-request events start a new box from the PR's head branch, when the event is about the environment's primary repository and the PR isn't from a fork.
  • New-box prompts are capped at CLOUD_AGENT_PROMPT_MAX_LENGTH.

Saving (automation/targetPlan.ts): planTarget(existing, input, lookups) decides every target column and which checks to run. The router does the lookups (a legacy workspace, a new pin's environment) and runs the checks: host access, cloud access, cloud agent (Claude or Codex), usable environment. Pins are limited to cloud workspaces the caller created, because waking a box uses its creator's credentials.

CLI / MCP:

  • automations create|update --cloud --environment <id> | --workspace <cloud workspace id>.
  • The environment is chosen the same way workspaces create chooses one: by id, with a default only when exactly one environment has repositories.
  • create says where runs will go.
  • update reads a cloud automation's --workspace as a cloud workspace, and refuses flags that only apply to a machine.

Desktop: picking Cloud swaps the sentence's pickers:

  • the environment replaces the project;
  • "using" lists your own cloud workspaces, with "new each run" as the default;
  • tags are hidden;
  • the agent picker offers Claude and Codex only.

Runs open their cloud workspace, and the automations list labels cloud rows "Cloud".

Manual QA

End to end on a local API against the worktree's Neon branch, with real Vercel sandboxes, the dev CLI and the dev desktop over CDP. Details are on the two pages above.

  • New box per run: dispatched in under 1 s. The box was ready in about 20 s, and the agent started with the automation's prompt and trigger context.
  • Pinned running box: 2–3 s. Continuing a session went into the same terminal.
  • Pinned sleeping box: woke and dispatched in 11.2 s.
  • Deadline, forced to 2 s: run failed as cloud_not_ready and named the box. The desktop toast shows the guidance.
  • Pinned box deleted: a new box started and the automation was re-pinned to it.
  • Refused cases:
    • a teammate's box, and a failed box;
    • a non-cloud agent;
    • host flags on a cloud automation;
    • an unknown environment;
    • continuing a session with no pin.
  • Desktop: switching between machine and cloud, the environment and workspace pickers, the session chip only when pinned, device-first order, save round-trips the right columns, runs open their box, the list shows "Cloud", and New Workspace with Cloud is unchanged.
  • Found and fixed while testing:
    • A box that never ran an agent refused agents.run.
    • The list labelled cloud automations "Session".
    • A failed run didn't name the box it was waiting on.
  • A scheduled run through QStash. QStash can't reach a local API; "Run now" goes through the same dispatchAutomation.
  • The real 30 s deadline expiring.
  • Cloud access revoked, or the owner leaving the org, between runs.

Testing

  • bun run typecheck: pass.
  • bun run lint:fix, bunx sherif --fix: clean.
  • bun run check:i18n: pass, with new strings translated for every locale.
  • bun run test: every package passes except @superset/desktop (34 failures). All are real-git, worktree and shell-environment tests that pass when run on their own (for example git.test.ts 39/39), and none touch code in this PR.
  • New: packages/trpc/src/router/automation/targetPlan.test.ts, 17 tests with no mocks, covering create and update target rules for host, cloud, switching between them, legacy retargets, and continue-session.

Migration: 0130

Statements:

  • ALTER TYPE … ADD VALUE ×3;
  • three nullable ADD COLUMNs;
  • three foreign keys;
  • one index on automation_runs(cloud_workspace_id).

Production sizes: automation_runs 106k rows (51 MB), automations about 1.4k, cloud_workspaces 76, environments 2.

Lock order: child tables first. There's no type rewrite, and no table in rule 4's hot list. The foreign-key check and the index build scan only automation_runs.

Not rehearsed under load. Under the updated skill it doesn't need to be: it touches no hot table and no large one. It applied cleanly on the worktree's copy of production.

Known limitations

  • A box's agents aren't set up until something lists them. Dispatch lists them before agents.run. The proper fix is for agents.run to do it itself, which needs a host-service release, and we still need to decide which agents a box offers (the image installs only Claude and Codex). Recorded as Open in docs/cloud-sandbox-mismatches.md.
  • Missing agent credentials fail silently. A box whose owner has no sign-in starts the agent at a login prompt, and the run still says dispatched. Already listed as Open in the considerations doc.
  • Boxes pile up. New-box-per-run automations leave one box per run, and there's no spend or rate limit. Per-org rate limits are the planned follow-up; noted in docs/cloud-sandbox-considerations.md.
  • Tags on a cloud automation are stored but unused.

Rollout / rollback

  • Rollout: migrations run first, then the API. Only accounts with the cloud-workspaces flag can save a cloud target; the server checks it on every save and every run.
  • Rollback: revert the code. The new columns are nullable and unused by the old code, and the enum values are harmless. Cloud automations then fail to dispatch, as they did before.

Summary by cubic

Automations can now run in cloud workspaces — each run starts a new workspace from an environment, wakes a pinned one, or continues the session the last run left in it. This works from the desktop, CLI, and MCP; previously choosing Cloud failed on save with "Host cloud is not registered".

Dispatch

  • Host and cloud dispatch now share one run lifecycle, and every run settles (none are left in dispatching).
  • The target rules for automation.create and automation.update moved into a pure, tested planner (targetPlan.ts); automation.ts is smaller than on main.
  • Pins are limited to cloud workspaces the caller created, since waking a box uses its creator's credentials.
  • A pinned box that's deleted, failed, or provisioning for over 10 minutes is replaced from the environment; a provisioning pin is marked failed before replacement.
  • The 30-second deadline bounds only getting ready (owner access, waking the box, listing its agents); sending agents.run or starting a box runs to completion under its own timeout. The dispatch route runs to 120 seconds so a slow host or cloud start isn't cut off mid-launch and left in dispatching.
  • Event payloads are trimmed so a cloud prompt always fits the launch limit; failing to fit no longer fails the run.
  • New error codes cloud_not_ready, cloud_access_denied, and cloud_environment_unusable carry desktop guidance; dispatch tells a PostHog outage apart from a revoked cloud flag, and only checks an environment that was named (a pin's environment is a fallback).
  • Pull-request triggers start a new box from the PR's head branch, but only for the environment's primary repository and non-fork PRs.

CLI, MCP, and desktop

  • CLI adds --cloud, --environment, and cloud --workspace; MCP accepts targetHostId: "cloud" with cloudWorkspaceId and environmentId, and create picks the environment like workspaces_create does. CLI update also refuses host flags, including --clear-tags, on a cloud automation.
  • Instructions too long for a cloud run are refused when saving; the desktop and CLI share that check.
  • The desktop sentence now reads device first; for cloud, the environment replaces the project, "using" lists the caller's own workspaces with "new each run" as default, tags are hidden, and the agent picker offers only Claude and Codex.
  • A pinned box is named from the organization's list, so a teammate's or provisioning box doesn't read "Workspace not found".
  • Runs open their cloud workspace, and cloud rows are labeled "Cloud" in the list.
  • Saving a cloud target is gated on the cloud-workspaces feature flag (checked on save and run); migration 0130 adds three enum values and three nullable columns and rolls back cleanly since the old code ignores them.

Known limitations: a box's agents aren't seeded until something lists them (dispatch lists before agents.run), missing agent credentials fail silently at a login prompt, and new-box-per-run automations leave one box per run with no spend or rate limit.

Summary by CodeRabbit

  • New Features
    • Create and manage automations that run in the cloud, choosing an environment or pinned cloud workspace from the desktop app or CLI.
    • Configure cloud automation agents, session continuation, and workspace placement; view and open cloud automation runs.
    • Receive clearer guidance when cloud automation setup or execution encounters access, environment, workspace, or agent issues.
  • Improvements
    • Automation runs can take up to two minutes.
    • Cloud automation instructions now follow a shared character limit.