feat(daemon): content protocol in the sandbox daemon

Draft
TL
4d ago
+7825-94

Part of the v8 Studio stack. Base: #7796. This PR folds the whole Go daemon track (#7812, #7813, #7814, #7815, #7847, #7817, #7818, #7819, #7820) into one PR.

The sandbox daemon now speaks the Deco content protocol over the working tree. The site editor can then edit a v8 (Blocks) site running in a sandbox the same way it edits a local deco serve. It is a Go port of @decocms/blocks/protocol.

PO decision, 2026-10-13: "Uploading directly to the fs should be done only on /site-editor play mode. Hosted mode (with a project etc) should use our s3 service, and sandbox has nothing to do with it." So the daemon takes no asset uploads: describe reports assets: null, and a sandbox session uploads to Studio's existing S3 asset service (POST /api/:org/file-configs/:id/upload, assets.decocms.com/<site>/…), like a hosted project. The upload route this PR used to add (PUT /_sandbox/assets/{name...}, was #7818/#7819) is gone (d6d03a7dd). This PR adds exactly one route: /_sandbox/rpc.

What it adds

All new code is in packages/sandbox/daemon-go/internal/content/ plus internal/routes/content.go:

  • JS-exact JSON (jsjson.go): number formatting and string escaping match JSON.stringify, so the blocks the daemon writes are byte-identical to the ones deco serve writes. golang.org/x/text moves from an indirect to a direct requirement. (was #7812)
  • Keys and errors (keys.go, errors.go): block keys map to file names with encodeURIComponent parity, the daemon finds every spelling a key may have on disk, and the protocol error codes are defined here. (was #7813)
  • Filesystem store (fsstore.go): reads and writes <app root>/.deco/blocks/*.json. Nothing can escape the tree. Commits take a cross-process lock (.deco/.blocks.lock), stage through .deco/.tx-*, and are durable and rolled back on failure. (was #7814)
  • Secret guard (secrets.go, ciphertext.go): plaintext is refused in v8 Secret fields and ciphertext must have the right shape. The v7 secret loader stays exempt. (was #7815)
  • RPC router and reads (rpc.go, handler.go, methods.go, params.go): envelope and batch handling, body limits, and describe, schema.get and blocks.list (with ifNoneMatch). (was #7847)
  • Writes: blocks.apply with normalization, static and schema validation, ifMatch, the commit plan and retry with backoff, plus the OnCommit hook. (was #7817)
  • No uploads: describe reports assets: null. The asset handler from #7818 was removed (see the PO decision above).
  • Mounted route (main.go, routes/content.go): /_sandbox/rpc sits behind the daemon token (d.authed). worktree.Lock.AcquireWithin is a bounded wait: if the client has already given up, the write never lands. (was #7819)
  • Conformance e2e (daemon-e2e/daemon.content-protocol.e2e.test.ts): runs the published @decocms/blocks conformance suite (its asset cases skip, since describe.assets is null) and a byte-for-byte parity check against the real daemon binary (the reference server reports assets: null too). @decocms/blocks is added as a sandbox devDependency; the only other change outside packages/sandbox/** is the matching bun.lock hunk. (was #7820)

Why one PR

The PO asked to release the hard-to-roll-back pieces first, at the base of the stack: #7796 (the org_sites migration), then this PR (the sandbox image). Everything else stacks on top. Shipping the daemon as one unit means one image bump and one revert if needed. The same code was already reviewed commit by commit in the folded PRs. Their history is kept in this branch, which is a normal merge of split/19-daemon-e2e-conformance.

Additive only

The PO's requirement is that the daemon only adds APIs and never removes or changes existing ones. Everything was checked against origin/main (21912beaf), whose packages/sandbox is identical to this PR's base.

1. HTTP routes. I diffed every mux.HandleFunc/Handle registration with grep -rhoE '(HandleFunc|Handle)\([^,]+' --include='*.go' --exclude='*_test.go'. Main has 31 and this branch has 32 (re-checked @ d6d03a7dd; the only diff line is HandleFunc(method+" "+pre+"/rpc"). The new one sits inside registerSandboxRoutes, prefix /_sandbox:

RouteMethodsAuthBefore this PR
/_sandbox/rpcGET POST PUT PATCH DELETE (anything but POST gets the protocol's 405)d.authed (daemon token)Fell through to the authed /_sandbox/ catch-all: 404 Not found

/_sandbox/assets/* is not registered: it still falls through to the authed 404 catch-all, as on main.

No existing registration was removed or changed (path, method, wrapper and handler are all the same). The new path doesn't collide with any fs route name (read write unlink mkdir rename edit grep glob bash), and OPTIONS preflight still goes to the existing corsPreflight. The only observable change: an authed request to /_sandbox/rpc used to get a 404 and now reaches the content protocol. Unauthenticated requests are refused exactly as before.

2. Exported Go API. apidiff skips internal/ packages, which is all of this module, so I dumped go doc -all for every package on both sides and diffed the dumps. 0 lines removed or changed. Lines added:

  • internal/content: a new package.
  • internal/routes: type Content, type ContentDeps, func NewContent(ContentDeps) *Content and (*Content).RPC. All live in a new file, routes/content.go.
  • internal/worktree: func (l *Lock) AcquireWithin(d time.Duration) (release func(), ok bool). Acquire() is unchanged.

The only pre-existing Go files touched are internal/worktree/lock.go, which gains AcquireWithin and a time import, and main.go. In main.go the diff is additions only (no - line): one field is added to the unexported sandboxHandlers struct, the loop above registers the route, and main() builds routes.NewContent from the existing repoDir, d.store, d.treeLock and fsDeps.OnWorkingTreeWrite.

3. Config, env and flags. I extracted every os.Getenv/os.LookupEnv/flag.* call: 24 on each side, identical. internal/content and routes/content.go read none, and there are no new CLI flags or config fields.

4. Behaviour of existing code paths. No diff hunk modifies the body of a pre-existing function, except the registerSandboxRoutes registrations and the main() wiring listed above. Specifically:

  • treeLock is still taken the same way by fs and git routes. The new code takes it only through AcquireWithin, and only for a content commit.
  • .git/info/exclude entries (/<app>/.deco/.blocks.lock, /<app>/.deco/.tx-*/) are added lazily, the first time a content route is used. They are never added at boot.
  • /_sandbox/decofile (v7) is unchanged. It reflects protocol writes because they're real working-tree writes reported through the existing OnWorkingTreeWrite hook.
  • The existing Go tests (main package, routes, gitx, …) pass unchanged.

Commands (run from packages/sandbox/daemon-go, with main extracted via git archive origin/main packages/sandbox/daemon-go):

for p in $(go list ./internal/...); do go doc -all -short $p; go doc -all $p | grep -E '^(func|type|var|const|    func|\t[A-Z])'; done > doc-<side>.txt
diff doc-main.txt doc-new.txt | grep -c '^<'            # 0
grep -rhoE '(HandleFunc|Handle)\([^,]+' --include='*.go' --exclude='*_test.go' . | sort   # routes
grep -rhoE '(os\.(Getenv|LookupEnv)\("[^"]+"|flag\.[A-Za-z]+\("[^"]+")' --include='*.go' --exclude='*_test.go' . | sort -u   # env/flags

Rollout and rollback

  • Rollout: this PR only adds one route, and nothing calls it until the web track (the content backend in the editor) lands further up the stack. A sandbox image built from this PR behaves exactly like today's for every existing caller.
  • Rollback: revert this PR and ship a new sandbox image. Live pods keep the image they started with, so nothing changes under a running sandbox. Old images answer the authed 404 for /_sandbox/rpc again.

Tests

  • go vet ./...: clean.
  • go test -p 1 ./... and go vet ./... in packages/sandbox/daemon-go: all packages pass, run locally @ d6d03a7dd (earlier tree: -race too).
  • bun install --frozen-lockfile: passes.
  • CI after removing the upload route (d6d03a7dd): Sandbox 38398232178 success, daemon-e2e 304 pass, 0 fail (305 tests, 16 files). The stack top (split/51, which contains this) also passed Checks & Unit Tests, E2E and Sandbox.
  • CI on the earlier tree (dispatched, because stacked PRs only get Gitleaks and cubic automatically) passed:
    • Sandbox 38251351755: daemon-e2e (305 e2e tests across 16 files, including the content-protocol conformance) and docker-smoke.
    • Checks & Unit Tests 38251356693.
    • E2E Tests 38251362384.
  • Local: the content-protocol e2e against a locally built daemon gave 79 pass, 0 fail.

Stack

#7796 (base, org_sites migration) → this PR (sandbox daemon) → API track (#7800 …) / web track (#7803 …) → e2e (#7841 → #7842 → #7843) → #7871 – #7876.

🤖 Generated with Claude Code

https://claude.ai/code/session_01WNwbSEePYNcY5YCgqZURig