feat(cloud): move sandboxes from Blaxel to Vercel

Merged
SA
2mo ago
+26907-1521

Why

Blaxel was acquired. Cloud workspaces now run in Vercel Sandbox. This also folds in #7391 (per-user agent sign-in); if that merges first, this branch's migrations need renumbering — if this merges first, close #7391.

What changes

  • Provider: packages/trpc/src/lib/sandbox/ (was lib/blaxel/). Persistent sandboxes with a 4h session timeout that only the open workspace extends; forks from a stopped golden's snapshot; the org's model keys injected at the egress firewall on image and fork sandboxes (a personal credential or environment variable for a provider suppresses its rule). Image built with vercel vcr build into the team's sandboxes project.
  • Auth: a Vercel sandbox's port is a public URL with nothing in front of it, so host-service is now the gate — the API signs a 10-minute Ed25519 token bound to one workspace id and the sandbox holds only the public key (SandboxAccessHostAuthProvider replaces the accept-everything EdgeGuardedHostAuthProvider). Clients present it as the same bearer a local host reads; all Blaxel preview-token plumbing (desktop + mobile) is gone. CSP allows *.vercel.run; host-service answers CORS * in sandbox mode.
  • Wake/promote semantics (measured, documented in docs/cloud-sandbox-mismatches.md): a resumed session has no processes; snapshot() ends the session and is the resume point; inbound traffic never wakes. Addressing and waking are separate requests (access reports running; only the open workspace wakes), the open workspace's address is held until host-service answers, and promote keeps the source resumable.
  • Migrations: 0112 agent_credentials (regenerated from #7391 after the 0111 collision) + provider defaults to vercel; 0113 retires Blaxel-backed rows (ready workspaces → failed, fork environments archived).
  • Desktop (third commit): TigerVNC's Xvnc replaces Xvfb + x11vnc (damage-driven, pushed frames — measurably smoother; the pane pins 1920×1200 scaled to fit). The image builds WhiteSur GTK/icon/cursor themes from pinned sources, ships the Big Sur Plank dock (Chrome, Files, Terminal bound to their own desktop entries — Plank matches windows by desktop-file name and reads GSettings, not its settings file), the Superset mark on the panel, photo wallpapers per workspace, JetBrains Mono terminal, and Chrome opening maximized on the window manager's frame with no first-run/crash bubbles and a CDP port. The desktop group installs with recommends (--no-install-recommends had dropped the SVG pixbuf loader). Boot: system bus, stale X lock removal on resume, xdpyinfo wait, bootstrap decision logged to /data/boot.log.
  • Toolchain (third commit): build-essential, clang, Python 3 + pip, JDK 17, Go 1.27.1 and Docker CE (fuse-overlayfs; dockerd starts every boot) from their vendors, gh, ffmpeg, tmux, htop, lsof and the rest of the reference desktop's list — verified on a fork: docker run -p answers, compose v5, gh 2.100, go 1.27.1.
  • Region: sandboxes live in VERCEL_SANDBOX_REGION (sfo1 — ~23 ms/request from San Francisco vs ~190 ms to iad1). Snapshots are region-bound, so forks inherit the golden's region and only image-created sandboxes take the setting; nearest-region routing needs a golden per region (follow-up).
  • Stop race: stop() resolves while the sandbox is still stopping; a fork taken before the stop snapshot is current boots from the bare image. Promote and the release now wait (waitForStopSnapshot) — this was the intermittent "fork lost node_modules" release failure.
  • Env vars (five places): VERCEL_SANDBOX_TOKEN, VERCEL_SANDBOX_TEAM_ID, VERCEL_SANDBOX_PROJECT_ID, VERCEL_SANDBOX_REGION, SANDBOX_ACCESS_SIGNING_KEY replace BLAXEL_*. scripts/sandbox/generate-access-key.ts mints the key.

Verification

docs/cloud-sandbox-acceptance.md is the suite, with results inline: 50 pass, 5 not run (two-fork independence, private-repo clone — no GitHub App key in dev, mobile simulator, the 3h extend path). Highlights, all against the real dev desktop over CDP and a real Vercel project:

  • bun run sandbox:smoke: auth gate (no / foreign / forged token → 401, valid admitted), CORS preflight *, root + natives, brokered Anthropic key → 200, branch, VNC handshake.
  • UI create → image workspace ready in 14s, Claude launched on the brokered key and answered; fork workspace ready in 6s with the golden's node_modules, display and zsh config.
  • Stop + reopen from the sidebar: sidebar polling leaves it stopped for 2 min; opening wakes it and the agent pane resumes with scrollback ~20–30s later; 35 min of token refreshes with zero 401s.
  • Promote via the API (source resumed after) → fork from the promoted environment inherits files with fresh identity and no promoter env; sandbox:release end to end in 93s.
  • Personal API key saved through the real API → real key in the agent's env, only OpenAI brokered; golden carries no credential.
  • Delete from the UI removes the row and the Vercel sandbox; nothing left in the project but the internal golden.

Third commit gate: lint clean; typecheck clean; check:i18n clean; turbo test green except the 4 pre-existing CLOUDFLARE_ACCOUNT_ID env failures in growth/* and plugins/oauth (files untouched by this branch). Desktop verified on a real fork over VNC screenshots: WhiteSur frames/icons/cursor, dock, logo, wallpaper, Chrome frame; boot log shows the fast bootstrap with baked deps.

First commits' gate: lint clean; typecheck clean for shared, trpc, host-service, desktop (mobile: only the pre-existing unref errors); check:i18n clean; bun test — shared 1012 pass, trpc 246 pass + 4 pre-existing sitemap.test.ts env failures, host-service 1712 pass + 1 pre-existing daemon integration timeout untouched by this change.

Before deploy

GitHub secrets (Production + Preview) for the API: VERCEL_SANDBOX_TOKEN (a Vercel access token for the sandboxes project, not the deploy token), VERCEL_SANDBOX_TEAM_ID, VERCEL_SANDBOX_PROJECT_ID, SANDBOX_ACCESS_SIGNING_KEY. Then sandbox:release --production. The BLAXEL_* secrets can go.

Follow-ups from the Linux desktop bench (2026-09-11 evening)

  • Chrome profile lock on wake: a woken sandbox restores Chrome's SingletonLock, which names the previous session's hostname, so the first launch stopped on "profile in use by another computer". start.sh removes the stale lock before the desktop starts.

  • App icon for the panel and dock: the icon theme entry named after the window class is what the panel button, Plank and the switcher show; it is now the real app icon instead of a black mark.

  • /dev/shm sized like a desktop: the VM mounts a 64 MB /dev/shm; Chromium's software compositor backs every window's tiles there, and a third Electron window on the sandbox display aborted its renderer with an out-of-memory SIGTRAP (minidump-verified). start.sh remounts it at 50% of RAM. Documented in docs/cloud-sandbox-mismatches.md.

  • Nerd Font in the desktop terminal: prompts from dotfiles print Nerd Font glyphs (git state, powerline) that plain JetBrains Mono rendered as diamonds; the image ships JetBrainsMono Nerd Font and the terminal uses it.

  • Workspace path for desktop shells: SUPERSET_WORKSPACE_PATH is exported for the desktop session, as host-service already does for the app's terminals, so dotfile navigation (nav r, gt r) resolves /workspace.

  • Desktop session advertised to shells: XDG_SESSION_TYPE, XDG_CURRENT_DESKTOP and DESKTOP_SESSION are exported with DISPLAY, so tools and agents that check for a display find one (an agent had started its own Xvfb next to the real desktop).

  • Image workspaces run 8 vCPU / 16 GB (the plan's cap): at 4 / 8 GB an agent bringing up the dev stack took the VM down.

  • The delete entry in the considerations doc was stale: cloud rows already route to cloudWorkspace.delete, verified against the provider.

Image rebuilt and released four times today; current golden env-internal-mtxi15ip (sfo1), probe passed (host-service, dependencies survive the fork, display + xfce + plank, VNC). Workspaces created before the release keep their old snapshot.

https://claude.ai/code/session_01WcGbULcxE9Nw1Y8YVqe3KZ

Summary by CodeRabbit

  • New Features
    • Added Agent Accounts settings for Claude and Codex subscriptions, API keys, and custom providers.
    • Added credential verification, secure storage, endpoint validation, and connection status indicators.
    • Added “Take control” and “Release control” actions for view-only desktop sessions.
  • Improvements
    • Cloud workspaces now run on Vercel-hosted sandboxes with signed access and improved wake/readiness handling.
    • Desktop and mobile sandbox connections use unified authentication.
  • Documentation
    • Added cloud-sandbox acceptance coverage and updated setup guidance.