PR #8006 - decocms/studio

What is this contribution about?

Studio maintains its own infrastructure manager to download PostgreSQL/NATS/MinIO, track processes and ports, and coordinate development startup. Replace that runtime with Compose Watch and direct server entrypoints, removing 2,773 net lines including the new configuration and documentation.

docker compose up --build --watch starts dependencies, explicit migrations, API and web. Source syncs into Linux containers; Bun restarts the API and Vite handles frontend reloads.
Reuse the self-host service definitions, persist JetStream data, gate startup on successful migrations and bucket initialization, and keep credentials/build outputs outside source synchronization.
Support isolated worktree stacks and optional shared infrastructure with separate databases, NATS accounts and buckets. Keep each worktree's frontend/API network private. Shared resources require manual provisioning with standard service tools.
Remove the embedded database, binary downloads, PID/state tracking, TUI and custom development/worktree launchers. Preserve client/scaffolding CLI commands and frontend-only development.
Update Helm, resilience/E2E startup and contributor guidance to use the direct runtime and explicit migration owners.

How did you verify your code works?

bun run verify --parallel=2 with CI-pinned Bun 1.3.14: 11,014 tests passed; formatting, lint and unused-code checks passed. All tests and per-file isolation remain enabled; only the worker count changed.
API, web and E2E typechecks; bun run check:readmes (14 workspaces).
Development image build and default/shared/release/test Compose configuration validation.
Live Compose checks: source synchronization for API/web, API process restart, updated Vite output, environment-file exclusion, signup/session cookies, successful initial migrations and nonzero migration failure on an unavailable database.
Live shared-infrastructure checks: separate PostgreSQL roles/databases and separate NATS accounts using identical subjects, JetStream stream names and KV keys.
Existing Helm validation scripts, API bundle build, and isolated installation of the npm-packed API package. The packed runtime loaded its dependencies and reached the expected synthetic database connection failure.

Unbounded local runs during laptop contention hit the NDJSON timed-flush assertion, the Monaco source-scan timeout and a ConnectionCard keyboard-test timeout. All three focused test files passed (16 tests), followed by the complete two-worker verification above. No test assertions, timeouts or exclusions changed. Packed-runtime checks preceded the main rebase; the subsequent local spike rebuilt and exercised the current Docker image. API/web/E2E typechecks and README checks passed earlier; the final follow-up changes only Compose, build-ignore rules and contributor guidance.

Full browser E2E and resilience/chaos suites were not run. Runtime checks used synthetic data; temporary test containers and volumes were removed.

Local spike

Combination	Result
Infrastructure only, then API, then web	Passed; API-only readiness 12.2s with existing image
Host API + host Vite against Compose infrastructure	Passed signup/session/HTML; preserve the same auth and encryption keys
Host Vite against existing container API frontdoor	Passed browser signup/session/reload; host Board follow-up incomplete
Container frontend	Passed browser signup/session/reload and fully rendered Board
Two application projects sharing infrastructure	Passed database, NATS Core/JetStream/KV and bucket isolation, private API routing, and independent teardown
API and worker roles	Both healthy; API listens to zero run queues, worker to seven
Source/image exclusions	Eight synthetic secret/generated-output sync probes excluded; fresh image excludes synthetic environment/key files

The spike reproduced and fixed insufficient capacity in the shared NATS example: its 4GB server cap could not initialize the second account's 4GiB output stream. The example now uses 16GB with documented headroom, and both streams were retested. Better Auth keeps its default cookie names across localhost ports; sharing sessions requires the same auth database and secret.

The follow-up used two actual Git worktrees with independent Compose Watch processes and one shared PostgreSQL/NATS/MinIO stack. Both real Chrome contexts completed signup and rendered the Board; concurrent authenticated reload took about 4.4s. Source edits and API reloads stayed within their own worktree, and distinct API/UI markers survived recreation with the default per-project image tags. Stopping A with down --volumes preserved B's container IDs, health, SQL, NATS KV and S3 state.

This caught ordinary shared-package files incorrectly triggering full rebuilds. The corrected manifest filter uses include: ["**/package.json"]; four existing/new-file probes now sync in 1.15–1.64s without changing any app container IDs/images or triggering a rebuild. Manifest and Dockerfile changes still rebuild only A while B remains untouched. Exact Dockerfile/ignore-file allowlist entries restore their explicit Watch triggers. Contributor instructions now require Compose 2.40+ for initial_sync (runtime spike used Compose 5.6.0).

Steadier follow-up snapshots after recreation: APIs about 269–395MiB each, Vite about 1.53GiB each, and shared infrastructure about 185MiB total. No app container OOMs were recorded. These remain observations, not a controlled benchmark; a pre-existing Kubernetes node was left running, and source/API reloads under contention were substantially slower. The manifest control took about 106s including dependency reinstall/image export; the Dockerfile control took about 29s. Docker reuses common build layers, while manifest changes can invalidate the dependency layer.

HTTP health alone does not prove agent-output stream readiness, so the spike explicitly checked both account streams. Real agent/provider sandbox execution was not exercised. Temporary containers, scoped resources, Git worktrees and owned host processes were cleaned up.

Screenshots/Demonstration

Development/deployment tooling change; runtime reproduction steps below.

How to Test

With Docker and Compose 2.40+, run docker compose up --build --watch from the repository root and open http://localhost:4000.
Sign up, edit API/web or shared-package source, and confirm API restart/frontend updates. Edit a dependency manifest or lockfile to exercise image rebuilding.
Stop with docker compose down, start again and confirm data remains. After adding migrations, run docker compose build migrate && docker compose run --rm migrate, then restart the API.
In another checkout, select a different COMPOSE_PROJECT_NAME and WEB_PORT. For server reuse, follow deploy/docker-compose/README.md; never share an application database or NATS account between divergent worktrees.

Migration Notes

Breaking: CLI server startup, deco dev, deco services, and the custom worktree/conductor launchers are removed. Use Compose/Kubernetes for packaged runtimes, or direct source commands against explicitly configured dependencies.
No new database schema migration. Better Auth/application and DBOS migrations run explicitly before startup; API source commands no longer provision dependencies or migrate automatically.
Existing CLI-managed data is neither imported nor deleted. Keep auth/encryption keys stable for persisted data. Compose uses normal authentication; loopback-only local auto-login cannot operate across container hops.
Shared NATS accounts use NATS_USER and NATS_PASSWORD together, or existing credential-file authentication. Account/bucket/database provisioning remains explicit.

Review Checklist

PR title is clear and descriptive
Changes are tested and working
Documentation is updated
No breaking changes — runtime CLI removals are documented above

Summary by cubic

Replaces Studio's managed local infrastructure runtime with Docker Compose Watch, removing the embedded PostgreSQL, binary downloads, process/port tracking, and TUI. docker compose up --build --watch now starts dependencies, explicit migrations, and the hot-reloading API and web; source syncs into containers while dependencies stay in the image.

Removes the CLI server commands (deco dev, deco serve, deco services) and the worktree/conductor launchers; the CLI now keeps only client and scaffolding commands.
Routes configuration through environment variables only; resolveConfig no longer takes CLI flags, and settings resolve lazily from env on first access.
Adds username/password NATS authentication alongside existing credential-file support for shared infrastructure.
Adds Compose overrides for isolated worktree stacks and optional shared PostgreSQL/NATS/MinIO with manual provisioning; JetStream data now persists via a volume. Better Auth keeps its default cookie names across worktree ports.
Scopes the dev image build so shared-package edits sync without full rebuilds; host-side Vite can target the API via STUDIO_LOCAL_API_TARGET.
Updates Helm, resilience/E2E setup, and contributor docs to run the packaged server directly and treat the migration job as sole schema owner.

Breaking changes

bun run dev, the managed CLI runtime, deco dev, deco services, and worktree/conductor launchers are removed; use Compose or Kubernetes instead.
API source commands no longer provision dependencies or migrate automatically; migrations and bucket initialization run explicitly before startup and block API boot on failure.
Existing CLI-managed databases are neither imported nor deleted; keep auth/encryption keys stable to retain access to persisted data, and use normal authentication since loopback-only local auto-login cannot cross container hops.