feat: Project Environments — Inspector UI (management, suites, journeys, runner)

Merged
CH
3mo ago
+7285-755

Project Environments — Inspector UI (PR 4, combined)

Exposes the Project Environments feature end-to-end in the inspector,
gated behind the single PostHog flag project-environments-enabled. An
environment is a named, project-scoped, live-editable bundle: one host +
optional standalone server group + optional pinnable skill selection. Suites
and journeys reference environments and resolve/pin them at run-start.

Combines the two work streams into one flag-gated PR (per the plan):

Part A — management surface + suites

  • New /environments management route (list⇄detail, archive/restore),
    editor (required host, optional server group, optional skills picker with
    P0.3 pinnability gating), suite multi-env picker (ordered, ≤10).
  • Run-all fan-out per environment (shared runGroupId); environmentId
    threaded through all four wire boundaries; prepareEvalRun resolves the
    closed server set via P0.1 resolveEnvironmentForLaunch before connecting
    servers / capturing the tool snapshot, and forwards
    expectedEnvironmentRevision to startTestSuiteRun.
  • Scheduled worker passes environmentId; run-detail "Environment" chip added;
    computer-env chip relabeled "Sandbox image". Public suite DTO exposes
    environmentIds (read-only).

Part B — journeys + swarm runner

  • Journey env-mode create/edit (ordered ≤10, compat hostIds recomputed),
    per-target runner (targetId), shared session-id mint
    (shared/swarm-session-id.ts), pinned-skill cache + two-channel harness skill
    delivery (host + environment provenance), per-target rollup/matrix, SSE
    targetId. Legacy journeys stay byte-identical.

Integration fixes in this PR

  • Aligned environment-launch.ts mirror with the actual P0.1 return shape
    servers:[{serverId,name}] (Part A had mirrored it as selectedServerNames);
    routed both callsites through environmentServerRefs.
  • Merged the flag hook + skill-types.ts cleanly (only 2 files overlapped A/B).

Backend prerequisites (must deploy first)

  • MCPJam/mcpjam-backend #768 — P0.1 launch resolver + P0.3 pinnability.
  • MCPJam/mcpjam-backend #769 — P0.2 two-channel skill composition.

Rollout gating

The flag gates ALL client exposure (sidebar, suite row, journey env-mode,
direct /environments nav). Server/runner protocol changes are strictly
additive and inert until an env-based suite/journey exists. Do not enable
project-environments-enabled until both backend PRs are deployed and the
runner deploy completes.

Verification

  • Full inspector suite: 9614 passed / 0 failures (6 skipped).
  • Client typecheck: clean. Server tsc: 82 errors = pre-existing broken baseline,
    zero new errors in touched files.

🤖 Generated with Claude Code

[!NOTE]
Medium Risk
Changes eval run fan-out, scheduling, and launch payloads (environmentId) with server-side resolution as the source of truth; misconfiguration or backend drift could cause failed or surprising runs until the feature flag and backend deps are aligned.

Overview
Introduces project environments behind project-environments-enabled: a new Environments nav item and /environments management flow (list/detail, create/edit with host + optional server group + pinned shared skills, archive/restore, revision-based conflict handling). Direct URLs redirect when the flag is off; writes use the same admin/owner gate as other project management surfaces.

Eval suites can attach ordered environments (up to 10). Run all fans out one run per environment via buildSuiteRunPlans, sending environmentId with empty client-side serverIds so the server resolves the closed set at launch; client server-readiness checks are skipped for these suites. Multi-environment schedules require picking a pinned environment. Run detail shows a flag-gated Environment chip from configSnapshot.environmentRef; computer Docker images are labeled Sandbox image to avoid naming collision.

Shared convexErrMessage replaces local Convex error parsing (e.g. computer environments drawer). Types and tests cover run plans, schedule pinning, orphan detach rows, and route flag gating.

Reviewed by Cursor Bugbot for commit 9a022c4831f841254b422570d7503c4e3de04123. Bugbot is set up for automated code reviews on this repo. Configure here.

Summary by cubic

Adds Project Environments to the Inspector: a flag-gated management UI plus suite, journey, and runner support for environment-based execution with pinned skills. Latest updates add orphan-selection handling in environment and skills pickers and reset the create editor on project switch, alongside prior runner guards, gating, and analytics improvements.

  • New Features

    • Environments management at /environments (create/edit/archive), with host + optional server group + optional pinned skills. Fully gated by project-environments-enabled; /environments is hidden from Atlas during rollout.
    • Suites: attach up to 10 environments; “Run all” fans out one run per environment; schedules can pin an environment; runs show an Environment chip (name + revision). Server resolves environments and enforces expectedEnvironmentRevision.
    • Journeys + Swarms: environment mode with ordered selection; per-target runner with targetId and shared session-id mint; rollups/matrix and SSE include targetId. Legacy journeys remain byte‑identical.
    • Pinned skills: picker forwards backend pinnability; runner delivers pinned artifacts to the emulated engine and harness via a small cache. Analytics report fan_out_axis and num_environments; toasts say “environment runs”.

  • Bug Fixes

    • Env launch wiring: mirror servers: [{serverId,name}]; use environmentServerIds (priming) and environmentServerNames (display). Hosted preflight converts sk_ keys; v1 public run route rejects environmentId.
    • Consistency: thread one resolved environment through hosted preflight and the scheduled worker into prepareEvalRun so expectedEnvironmentRevision matches the connected set (clean 409 on post‑edit).
    • Runner reliability: pinned‑skill cache coalesces fetches and survives a coalesced caller abort; abort during prefetch no longer marks attempts failed; stricter host‑skill‑channel guard; PinnedSkillArtifact validates description; harness path handling prunes stale files and skips sandbox‑wide prune when already aborted.
    • UI/UX and gating: kill‑switch hides all env surfaces (run‑detail chip, schedule pin, journey env targets); SwarmsTab env query gated by shouldQueryProjectId; selected‑but‑archived envs render as detach‑only; selected‑but‑missing ids (orphaned) now render as detach‑only in both the suite env picker and the environment skills picker (including when no shared skills are listed); schedule writes serialize through one apply path; project switch resets the environment editor (including open create form); skills picker keeps selected‑but‑ineligible rows for repair; env hooks normalize projectId; PinnedSkillArtifactFile enforces “exactly one of content|base64”.
    • Display and live updates: live SSE cell keys collapse host‑shaped targetId to bare hostId; journey list/matrix avoid truncated host names; draft seed effects are content‑keyed to avoid losing unsaved env selections; Sessions/Turns render exactly once; single‑case quick‑run guards env suites with a “Run all” message.
    • Access: allow anonymous Convex owners to manage environments (mirrors SwarmsTab).