feat(plugins): plugins and their MCP tools work in a cloud workspace

Merged
HA
1mo ago
+921-133

Summary

  • A cloud workspace provisioned zero skills — not zero plugin skills, zero. getBundledPluginDir looks for templates/plugin and falls back to a repo checkout, and the host tarball shipped neither. It now ships the bundled plugin and the installable first-party trees.
  • The plugin ledger on a box comes from sandbox.conf: claim.ts reads the creator's installs, host-service turns them into installed_plugins.json before provisioning runs. No pull, no account call, no second sync pass.
  • The MCP half: a box's credential is accepted on the plugin proxy only, rate-limited per workspace, and one ledger-driven writer in agent-setup keeps mcpServers in step with the skills.

Reviewed against Plugins in Sandboxes: Review. Deltas from it are in Design Decisions.

Why / Context

superset plugins list --json in a cloud workspace returned A cloud workspace cannot call plugins.list. That error was the symptom of three separate gaps, which is why this is not a two-line change:

  1. No skills at all. getBundledPluginDir() resolves <templates>/plugin, then falls back to <repo>/plugins/superset — which exists in a dev checkout and on no box. Every cloud workspace logged Bundled plugin missing …; skipping managed skills and provisioned nothing, including the superset:* skills.
  2. No ledger. installed_plugins.json is per-machine and its only writers are the desktop main process and the CLI. Nothing in the sandbox image or host-service ever wrote it.
  3. No tools. /mcp/plugins/* goes through resolveMcpContext, which accepts an API key or an OAuth JWT. A box holds neither, so every plugin tool call was a 401.

How It Works

The claim writes, the box reads. creatorPlugins() reads the creator's plugin_installs and claim.ts writes SUPERSET_SANDBOX_PLUGINS into sandbox.conf beside SUPERSET_SANDBOX_REPOSITORIES — on create and on every wake, because a wake is a re-claim. seedSandboxPlugins() runs in serve.ts immediately before provisionAgentIntegrations(), so the boot-time sync that already existed finds the ledger on disk. The box never asks the API for account state.

No clone in the box. Every account install is first-party (plugins.install has one insert site and refuses any other marketplace), and all of plugins/ is 244 KB, so produce.ts ships the trees and the seed points installPath at them. There is no git in this path.

One writer for mcpServers. desiredPluginMcpServers + syncPluginMcpServers live in agent-setup and run in the same pass as createManagedSkills, off the same ledger. The desktop's local-db-driven copy is deleted; its mutations already shell out to the CLI, which now converges both halves.

The sandbox credential on one route. resolveMcpContext takes an opt-in sandboxCredential flag that only the plugin proxy passes. The resulting context carries bearerToken: "" on purpose: nothing in proxy/ reads it, but anything reaching hosts through the relay does, so Superset's own tools fail on a box rather than acting as the creator. apps/api/src/app/mcp/route.ts also refuses source: "sandbox" outright.

Manual QA Checklist

Verified in a real cloud workspace (box-side)

  • Baseline in a fresh box: ls ~/.agents/skills is 0, no plugins key in /etc/superset/sandbox.conf
  • With the conf key present: ledger written from the conf alone, 16 skill dirs (13 superset:* + 3 linear-*)
  • installPath resolves to a shipped tree; ledger records the tree's version (1.5.2), not the conf's (0.0.0)
  • A disabled install stays a record and materializes nothing
  • A tools-only plugin with no tree (circleback) still gets its MCP server
  • MCP entries: linear → api.superset.sh/mcp/plugins/superset/linear, circleback → its direct URL
  • superset plugins list from inside the box still refuses — production API is pre-deploy, which is the expected state

Verified in a Linux container (11/11)

  • The same chain with real seedSandboxPlugins + createManagedSkills + syncPluginMcpServers, no network
  • Login-shell SUPERSET_HOME_DIR before/after the rename: /home/ubuntu → /home/ubuntu/.superset

Not verified — needs a deploy

  • plugins.list from a box returns 200 (needs the api deploy)
  • A plugin tool call from a box succeeds (needs the api deploy)
  • claim.ts writes the conf key on a real create and wake (needs the api deploy)
  • boot-twice (needs native amd64; the 8 new assertions cover this chain)

Testing

  • bun run typecheck — 42/42 tasks
  • bun run lint — clean
  • bun run check:plugins — clean
  • bun run check:i18n — no catalog changes (no translated strings added)
  • bun test packages/host-service/src/runtime — 221/221, including 5 new sandbox-plugins tests
  • bun test packages/sandbox — 9/9 · bun test packages/cli/src/lib/plugins — 46/46
  • 8 new assertions in boot-twice.ts covering the ledger, a treeless plugin, installPath, skills, the bundled skills, the mcpServers entry and the login-shell SUPERSET_HOME_DIR

Pre-existing failures, unchanged by this branch: 5 notify-hook "agent identity precedence", 22 rrule describeSchedule (Lingui macro outside compilation), trpc/credentials.test.ts (no local .env), and 20 desktop host-service-coordinator/notifications/host-db-workspace-name tests (no test under main/lib imports plugin-installs).

Design Decisions

  • SUPERSET_HOME_DIR → SUPERSET_USER_HOME in the contract. The contract's name meant $HOME while meaning ~/.superset in agent-setup, the CLI and the desktop. host-service exports the latter into every terminal; a login shell then sources contract.sh, and because the variable is already exported the reassignment is exported too. The CLI's ledger landed in $HOME, split from the one provisioning reads. Demonstrated before/after in a container. All 9 contract-side uses meant "the user's home", so the contract is the one that moved.
  • Queries key on user_id alone, not organization_id + user_id as the review specified: plugin_installs.organization_id is nullable and the install path writes it null, so narrowing by organization matches nothing.
  • SUPERSET_SANDBOX_MARKETPLACES dropped. With first-party trees shipped and account installs first-party-only, nothing on a box resolves a marketplace, so the key would always be empty.
  • Tools-only plugins stay in the ledger without an installPath. A catalog entry can have a direct MCP server and no tree (circleback); dropping it cost it servers that have nothing to do with a tree. Skill provisioning requires the field, MCP provisioning does not.
  • bearerToken: "" for a sandbox context rather than a minted JWT, so the guarantee does not rest on the opt-in flag alone.
  • plugins.list and plugins.marketplaces.list stay on the sandbox allowlist, against the review's "drop these two rows". superset plugins list and superset plugins marketplace list working inside a box is the point of this PR. Both are reads; every account write (install, uninstall, enable, disable, connect) stays sandbox: false.
  • The review's CLI sandbox: false work is already on main (#7776 landed the second axis); nothing to do there.

Known Limitations

  • plugins install/uninstall/enable/disable/connect stay hidden in a box (sandbox: false); account writes remain a person's call from a signed-in client. A plugin with no connection returns PLUGIN_UNAVAILABLE.
  • A box serves the plugin version its runtime ships, not the version the account records.
  • packages/agent-setup/** is not in .github/workflows/sandbox.yml paths, so a future agent-setup-only change to this chain gets green CI without boot-twice. Worth adding; left out to keep this diff to one concern.

Risks / Rollout

  • Risk: the sandbox credential reaching Superset's own MCP tools would hand a box the creator's full tool surface. Mitigated three ways: the capability is opt-in per route, the plugin route is the only caller, and /mcp refuses source: "sandbox".
  • Risk: before the per-workspace rate-limit key, bearer-less boxes behind shared egress would have shared one 600/min bucket fleet-wide. Ships with the auth change for that reason.
  • Rollout order: api deploy first (allowlist, MCP auth, rate-limit key, and the claim writing the conf key), then a bundle release (bun run release) for host-service, agent-setup and the runner. A box with no conf key simply seeds nothing, so the API landing first is safe.
  • The release must push the image. superset-boot is a rootfs file and the runner that handles a boot is the one already on the box, so the new host_env entry reaches an existing box on its second boot after the bundle lands, and a fresh box only through the image. Do not use --skip-image.
  • Rollback: revert the api deploy to stop the new auth path; point environments.bundle_sha at the previous sha (environment.update) and the next wake flips current with no download. Nothing here migrates data.

Summary by cubic

Cloud workspaces now provision the creator's plugins and their MCP tools, including the creator's connections; before, they provisioned zero skills.

  • The claim always writes the install set and the creator's connections to sandbox.conf; host-service seeds the plugin ledger and connections.json from them before provisioning runs, so a connector with two accounts gets split MCP entries on a box. installPath resolves to a cached copy of a bundled first-party tree — the full tree, not just skills — staged and renamed into place so a partial copy is never recorded; a same-named plugin from another marketplace or a missing tree stays tools-only, and an empty set empties the ledger.
  • Plugin MCP servers sync off the same ledger in host-service and the CLI (readEnabledPlugins); there is now one reconciler rather than two competing ones, same-named installs from other marketplaces are never served Superset's endpoint, an unreadable ledger counts as unknown rather than empty so a torn write cannot reap every managed server, and a server the reconciler wrote that the ledger no longer names gets reaped while a server the user configured themselves is untouched. The desktop keeps its local-db-driven sync because its plugins.syncConnections route calls it.
  • A box's credential is accepted only on the plugin MCP route; the MCP rate limit is keyed on the credential's verified workspace id (HMAC, no database).
  • The contract's SUPERSET_HOME_DIR is renamed SUPERSET_USER_HOME.

Rollout

  • Deploy api first, then a bundle release that must push the image.

Summary by CodeRabbit

  • New Features

    • Sandboxed workspaces can now receive enabled plugins from the workspace creator’s plugin setup, including bundled plugin tools and MCP servers where available.
    • Sandbox credentials are supported for MCP connections, and sandbox access now includes plugin and marketplace listings.
    • Plugin enable and disable actions are reflected in the bundled CLI.
  • Bug Fixes

    • Rate limits for valid sandbox credentials are now applied per workspace.
    • Plugin installation and setup handle missing or unusable plugin data more safely.