feat(environments): expose Project Environments on the API, SDK, and CLI

Merged
CH
3mo ago
+6841-1006

Project Environments have existed in the backend and the hosted UI since #3405, but had zero public surface — and the v1 eval-run route actively rejected environmentId. This adds the whole surface and makes environment runs work end to end.

Pairs with MCPJam/mcpjam-backend#783. Both repos keep a byte-identical copy of the v1 error contract, guarded by golden fixtures. Merge that one too — each repo's fixtures are self-consistent, so neither breaks alone, but they'll drift until both land.

What a Project Environment is

A named, project-scoped, live-editable execution bundle that eval suites and journeys run against: one host + optional standalone server group + optional pinned skill selection + optional pinned plugin versions. Revisioned for optimistic concurrency.

Not a Computer sandbox image (/images, renamed in #3418) and not the eval-suite environment servers bag. Nothing named images / PlatformImage* / computerEnvironments:* was touched.

Changes

v1 routes — new server/routes/v1/environments.ts:

RouteNotes
GET /projects/:p/environments?includeArchived=true — needed to find something to restore
POST /projects/:p/environments201
GET/PATCH …/:environmentIdPATCH requires expectedRevision
POST …/:environmentId/archive | /restoreexplicit sub-actions, not DELETE
GET …/:environmentId/resolvelaunch preview

Archive/restore aren't a DELETE because the backend keeps the row and restore has real semantics — it re-checks the name and drops irrecoverably-dead plugin pins. Routes stay off the guest allowlist. Reads need project membership; every write needs project admin, mapped to a real 403 (while non-membership stays a 404, so project existence never leaks).

Eval runs now accept environmentId. This was the gap that mattered most — without it the API could create environments it couldn't use. The route resolves the environment before building the manager, connects its closed server set (including servers contributed by pinned plugin versions), and passes that same resolution to prepareEvalRun so the run is pinned to the revision whose tool snapshot was captured.

SDK — PlatformEnvironment* types, PlatformApiClient methods, and seven MCP operations (list_project_environments, get_, resolve_, create_, update_, archive_, restore_). Not added to PLATFORM_CATALOG_OPERATIONS (the guest worker surface), matching hosts and images.

CLI — mcpjam environments {list,get,resolve,create,update,archive,restore}.

OpenAPI + docs — full spec entries (the openapi-drift test requires them), CLI reference, overview, and public-api.mdx.

Two contracts surfaced rather than smoothed over

expectedRevision is required on every mutation, not re-read server-side. Silently supplying the current revision would convert the backend's safe concurrent-edit rejection into a last-write-wins clobber.

PATCH keeps the tri-state: omitted = unchanged, explicit null = clear, value = set. Empty arrays stay rejected — they are not a way to clear. Both directions are covered by tests.

Error mapping

Errors branch on the structured ConvexError.data.code rather than sniffing messages. Note CONFLICT is overloaded backend-side — stale revision, duplicate live name, edit-archived, already-archived, not-archived — so all map to 409 and the backend's own message is surfaced instead of us asserting a cause. The /resolve endpoint maps the ENV_* family, keeping the specific code in details.code.

This required adding CONFLICT (409) to the v1 public contract, which had no code for "well-formed request, wrong resource state." That also fixes a live bug: ENVIRONMENT_REVISION_CONFLICT was missing from INTERNAL_TO_V1_CODE, so run-start revision conflicts were already reaching API callers as a 500.

Verification

  • Full suite: 9,734 passed, 0 failed (2 files skipped).
  • Server typecheck: 85 errors vs. the 88 pre-existing baseline — diffed, zero new. The three that vanished were missing-bundle artifacts generated as a test prerequisite, unrelated to this change.
  • typecheck:client and tsc -p cli/tsconfig.json: clean.
  • openapi-drift: green.
  • New environments.test.ts: 29 tests — guest denial, DTO shape (no environmentId leak), includeArchived forwarding, tri-state PATCH, revision 409, admin-403 vs member-404, archive/restore preconditions, and ENV_* resolve mapping.

Pre-existing Prettier violations in sdk/src/platform/errors.ts, show-servers.ts, and one block of v1/evals.ts were left alone; only my own lines are formatted.

Not in this PR

Journeys still have no public run path — the swarm runner is web-only and takes no environmentId. Worth a follow-up.

🤖 Generated with Claude Code


[!NOTE]
Low Risk
CLI and documentation only; delegates to existing SDK platform operations with no server or auth logic changes in this diff.

Overview
Adds mcpjam environments so hosted projects can be managed from the terminal: list, get, resolve, create, update, archive, and restore project environments (host + optional servers, skills, and plugin pins — distinct from Computer images).

The new command module follows the same platform CLI pattern as hosts: --api-key / --api-url, SDK operation execution with timeouts, Zod validation, and --file / --json bodies where flags do not cover structured fields. Mutations require --expected-revision from get, with local validation so bad revision strings fail before the API. Create/update merge JSON with CLI flags (flags win); update documents clearing pins via explicit null in JSON.

cli/src/index.ts registers the command group. Docs add the environments row to the CLI overview and a full reference section (revision workflow, admin-gated writes, pinning rules).

Reviewed by Cursor Bugbot for commit b62b4fba667532c5193a1b9e604a0128cdae344f. Bugbot is set up for automated code reviews on this repo. Configure here.


Summary by cubic

Expose Project Environments on the public surface (v1 API, @mcpjam/sdk, and @mcpjam/cli) and allow eval runs to target an environmentId. Also fixes name-based selection across live vs archived environments and updates OpenAPI so environment-backed runs validate correctly.

  • New Features

    • v1 API routes: list/get/create/update/archive/restore plus /resolve; supports ?includeArchived; archive/restore are sub-actions.
    • Eval runs: accept environmentId, resolve environment and connect its closed server set, pin run to the resolved revision.
    • SDK (@mcpjam/sdk): PlatformEnvironment* types and operations (list/get/resolve/create/update/archive/restore).
    • CLI (@mcpjam/cli): mcpjam environments {list,get,resolve,create,update,archive,restore}.
    • Access: not on the guest allowlist; reads require project membership, writes require project admin.
  • Bug Fixes

    • Add CONFLICT (409) to the v1 error contract; map ENVIRONMENT_REVISION_CONFLICT to 409 instead of 500. Enforce optimistic concurrency: all mutations require expectedRevision; PATCH keeps omit/null/set tri-state.
    • Environment selector: prefer live names for all ops (restore prefers archived), exact ID wins, and ambiguity is reported correctly.
    • Eval runs with environmentId: ignore caller serverIds, connect the environment’s closed set (plugin-contributed servers included), pin the resolved revision, map launch-resolution failures to 409 with ENV_* in details, and return 404 for unreadable environments.
    • OpenAPI: allow new-suite bodies with suiteName + tests + environmentId (no serverIds), and refresh request descriptions.