PR #7887 - decocms/studio

Server side of Blocks v8 (part 1 of 2). The editor that uses it is the stacked web PR, which needs this one merged first.

This PR and #7888 replace the v8 stack (#7877, #7800–#7825, #7803–#7840, #7841–#7843, #7871–#7878), whose PRs are closed with a link here; their branches are kept. The order is this API PR, then the web PR #7888.

PO decisions this implements

The sandbox daemon only does fs and git; the rest of the APIs live in Studio's backend. The daemon is unchanged (git diff origin/main -- packages/sandbox is empty). #7877, the Go content protocol, was closed as superseded.
v8 is decided per site, from main's schema. isV8Schema in @decocms/shared/blocks-major returns true when .deco/schema.gen.json has a top-level "blocksMajor": 8. There is no org flag.
Staging and prod share the site-token signing key and the R2 delivery bucket. There is no kill switch: a site leaves v8 when its main schema goes back to v7.

What's in it

packages/shared

blocks-major.ts is the one v8 detector. Its guard test fails if any other file decides v8 or reads meta.gen.json for that.
session-runtime: a v8 project that has a preview server always gets Fast Preview. v7 keeps its switch.
secret-ciphertext.ts: the sealed-box format shared by the server and the editor.
site-editor is now a reserved org slug.

Content protocol over GitHub (decofile.ts, decofile/repo-content-storage.ts)

POST /api/:org/decofile/:id/:branch/rpc serves the Blocks content protocol (createContentHandler from @decocms/blocks@8.1.0) over the repository. It also serves hosted draft publish and its status.
A v7 main gets a read-only answer with schema: null.
v8 gates are computed from main and never trust the marker. A site owned by another project gets 409 site_not_owned. An unconfigured bucket gets 503 hosted_delivery_unconfigured.

Content protocol in a sandbox (sandbox/content-storage.ts, sandbox-proxy.ts)

/api/:org/sandbox/:id/:branch/rpc runs the same protocol core over the daemon's existing authed routes: read, write, unlink and bash. There is no proxy to /_sandbox/rpc. A session without a sandbox gets 404.
Listing: one bash call per listing. Files that git diff-files sees as unchanged take their hash from git's index, which is git's own path + mtime + size check (racy entries are re-checked by content). Only changed, untracked or filtered files (autocrlf, filter, eol, working-tree-encoding) are hashed with git hash-object. On a 3,000-block repo with 2 dirty files, a listing takes about 0.1 s.
The listing goes through a file, not stdout. The command writes it to a folder in the sandbox's tmp and prints one line; Studio reads the file through /_sandbox/read. The file ends with an @end line. A missing @end, a malformed line or a truncated bash answer is refused as Unavailable, so a cut listing never pairs a hash with the wrong file or reads as an empty site. Each listing removes the previous one's folder; folders left by a pod that went away are swept after 5 minutes.
Reads: parallel, 8 at a time, and only for versions not already cached. Each sandbox keeps a warm handler (LRU of 16 sandboxes) with an 8 MiB body cache.
Apply: serialized per sandbox. Each apply takes a fresh listing and checks every expected version and the schema's. A mismatch gives the protocol's 409 Conflict and retry. Then the writes and deletes run in order and the listing cache is cleared. On failure every touched file is put back, including a write that landed but then timed out.
Edits right after a poll: the listing is reused for 250 ms. A block body that disagrees with that listing drops it, so a poll right after an agent edit reads the edit and does not fail with "kept changing".
Remaining race: another writer can still slip in between the check and the write. That means the agent, or another API pod, since the lock is per process. The daemon's own file routes have the same exposure.
Remaining cost, for the PO: each listing is one daemon task, so about one "task exit" log line and two "tasks" SSE broadcasts every 2 s per open editor. Its output is about 120 bytes. Only a daemon stat or listing route would remove this, and decision (1) rules that out.
Tests: the published conformance suite, the byte-for-byte parity run against the reference fs server, and the sandbox cases: outside edits, conflicts, a sub-folder app root, no .deco folder, rollback, cut listings, and index versus hashed versions (committed, staged, same-size edits, new, deleted, CRLF). They run against an in-process stand-in for the daemon in the unit tests, and against the real main daemon in content-storage.daemon.e2e.test.ts, a new step in the Sandbox workflow.

Hosted Deco CMS (hosted/, /api/:org/hosted/:id/{releases,site-tokens})

The delivery bucket (R2 through its S3 API) and its CDN purge, drafts, releases (list, and reapply an earlier version: see below), publish, and Ed25519 site tokens.
Config: DELIVERY_R2_*, DECO_SITE_TOKEN_SIGNING_KEY and CF_DELIVERY_ZONE_ID/CF_PURGE_API_TOKEN. When these are unset, the routes answer "not configured" and the purge is skipped with a warning.
The blocksMajor marker: it caches main's isV8Schema answer on the project. It is written only by code that has just run the predicate: create/import, thread create and the hosted gates.
Admin backfill: POST /api/_admin/hosted/site-claims/backfill is a dry run by default. It links existing v8 projects to their org_sites row and syncs the marker.
Project create: a free site slug is claimed for the new project's org. It never takes another org's slug, a deleted org's, or one deco.cx has. A linked site gets its managed asset config (S3); the deco.cx import shares that code (file-storage/managed-site-config.ts).
Thread create: a new thread on an unmarked repo-backed project checks for v8 first, bounded in time, so a v8 site gets the cms runtime.

Releases: Reapply version rolls forward (PO decisions 2026-10-14)

Releases never move latest.json back to an older commit. The pointer flip (makeCurrent) is deleted, and there is no kill switch.

POST /api/:org/hosted/:id/releases/:sha/reapply (hosted/reapply.ts) makes a NEW version whose content equals that version's.
- It commits to main, restoring .deco/blocks/*.json from that version's commit:
  - A block changed since is copied back (copyFromRef: exact bytes, nothing uploaded).
  - A block added since is deleted.
  - An equal block is not touched.
  - schema.gen.json, index.ts and the rest of the code stay as they are. A tracked blocks.gen.json is regenerated, as Publish does.
- Then it runs Publish's release path, now shared as releaseCommit(): companion release, latest.json, purge. It has the same main-moved check as Publish (409 main-moved).
- If main already has that content, the answer is {result: "up-to-date"} and nothing is committed. A commit with no release gives 404.
Schema change (round 3): a version made for another schema is reapplied anyway. There is no 409 and no second confirmation.
- The answer adds schemaChanged: boolean, which is logged as info.
- GET …/releases/:sha/reapply → {schemaChanged} lets the dialog know before the user confirms. It compares the blob ids of schema.gen.json at both commits and reads the schemas only when the ids differ. Canonical JSON means a formatting-only change doesn't count.
- The new commit keeps main's schema, so it is served with the current code. Going back is another reapply.
Message: follows git's revert convention: Reapply: <original note>.
- It never chains: reapplying a reapplied version keeps a single prefix.
- A version without a note gets Reapply: version from <YYYY-MM-DD HH:MM> UTC.
- A Reapplied-From: <sha> trailer goes in the same block as the co-author, so GET releases returns reapplyOf: {sha}.
No served-revision reads: GET releases returns current (the latest.json pointer), the commits and the cursor. There is no telemetry or stats-lake query; decocms/stats-lake#64 was closed unmerged.
POST …/releases/retry (releaseMainHead) is Publish's "Try again" (it was releases/current, which is gone).
Git providers: content clients gain readCommit(sha) (GitHub, GitLab, Bitbucket). Insights commits carry body (GitHub messageBody).
Tests:
- reapply.test.ts: changed/added/deleted blocks, only differing files touched, code untouched, gen artifact, empty diff, main moved, no release, schema changed (reapplied with schemaChanged: true), formatting-only schema change, message including chained and no-note cases, a later edit on top.
- releases.test.ts: reapplyOf, releaseMainHead.
- hosted-releases.test.ts route tests: reapply, the GET check, no 409 on a schema change, retry, and releases/current is 404.
- v8-gates.test.ts: the gates.

With this PR alone

Nothing in apps/web calls the new routes yet, and web is unchanged: it typechecks against the new packages/shared.
v7 projects behave as on main, with two exceptions:
- The managed assets' public base now defaults to https://assets.decocms.com (it was https://decoims.com).
- The new-thread check above can add one main head read, ETag-cached, when a repo-backed project with a preview server has no marker and Fast Preview off.
The releases sidebar view and the web halves of the detector guard ship with the web PR. They need web code to exist.

Rebased on main f3dffbf3e (2026-10-16)

The branch is rebased on origin/main at f3dffbf3e (106 commits past the last merge, 5015180cd). The three old merges of main are gone; the 15 commits of this branch are replayed on top of main, one by one, with the same subjects. The pre-rebase history is on backup/fold-v8-api-2026-10-16. The final tree equals a plain merge of the old head with main: the only conflicts were apps/api/Dockerfile (main's #8006 starts the server by path, so this branch's CLI-by-path change is dropped and deco-bin.test.ts now checks that the image starts Studio by path, never through the deco bin) and bun.lock (regenerated). In the first commit, commit-coalescer.ts takes the resolution of the old merge: the shared gen-artifact.ts helpers with #7909's accepted set/del. Main has not touched apps/api/src/decofile since 5015180cd, so #7909's server behaviour (the PATCH route's expect and 409 block-conflict, the per-block fingerprint check, the value-by-value merge on sync) is main's, unchanged. The v8 counterpart of #7909 (stale section saves refused through the content protocol's ifMatch) is in #7888.

Who made the other change, and when: hosted v8 only (PO 2026-10-16)

PO (final): "only have metadata and changed by on v8 hosted".

Where it lives: in the hosted draft object's storage metadata, never in its JSON. The draft stays exactly { set, delete }, no block carries anything, and the site's SDK never sees it.
Format: x-amz-meta-deco-edited on the draft object: entries joined by ;, oldest first. Each entry is <name>,<userId>,<seconds>. The name and user id are encodeURIComponent-escaped so the value stays ASCII, as a header must, and the time is in base-36 Unix seconds. The value is capped at 1900 bytes, under the 2 KB of user metadata S3 and R2 allow per object; past that the oldest entries are dropped first.
Writes: DraftStore.update stamps the names a save changed with the session user (the ref's editor). It keeps the others, drops names no longer in the draft, and writes the map with the draft. It carries the map forward from the GET the save already makes, so there's no extra HEAD per save.
Reads: load decodes the map from the same GET. The session GET answers changedBy: { [name]: { userId, at } } next to { draft, version }. The editor resolves names and avatars from the org's members.
Store support: DeliveryStore.putJson takes metadata and get returns it.
Fast Preview (v7) and Localhost: no who/when. The git-history route (changed-by), the trailer and author parsing, insights' path filter and the block __meta experiments are reverted in a06d99e1a.
Tests:
- draft-editors.test.ts: encode and decode with any characters, malformed values, the cap dropping the oldest, and stamping.
- draft-store.test.ts: the map carried forward across saves, one object, JSON free of ids, one put per save, discard and publish.

Checks

Who and when on hosted v8 (a06d99e1a reverts e3544e9b7, 92038d914, d3df85b4c; 0d8ce2bdf the draft object's metadata, 2026-10-16): apps/api typechecks (tsc, single-threaded); oxlint and biome are clean. The hosted, decofile, v8-gates and decofile route tests pass (318). The tree outside apps/api/src/hosted and the session GET equals 3914dd50e.
2026-10-16: this branch is unchanged at 3914dd50e. The web half #7888 is at 10684efe8 with the concurrent-edit fixes from the spike; e2e on its production build (this API included): decofile-api, fast-preview-git-sync, content-protocol-deco-serve and site-editor-stale-form pass.
After the rebase on f3dffbf3e: apps/api, packages/shared and apps/web typecheck (tsc, single-threaded), oxlint and biome are clean. Unit tests: the api, migrations and shared suites 5946 pass, 1 fail (heartbeatWhileQueued in hosted-harness-workflow.test.ts, a timing test this branch does not touch; it passes 2 of 3 runs alone); the web suite 4771 pass, 0 fail.
Before the rebase: the branch includes origin/main at 5015180cd (merge af8b74dde). After that merge: the decofile, hosted and route tests 335 pass, packages/shared decofile tests 49 pass (including block-merge.test.ts), the api and shared unit suites 5882 pass (7 fail only when run in one process, from mocks leaking across files; each passes alone), apps/api, packages/shared and packages/e2e typecheck, oxlint, biome and knip are clean. Locally, e2e decofile-api.spec.ts (19) and fast-preview-git-sync.spec.ts (3) pass.
CI on this PR at 08eb358bf is green: Checks & Unit Tests, E2E, Sandbox (including the daemon content step), Storage Integration, SAST and CodeQL. Multi-Pod failed twice on the PR run in the rollout churn scenario, at two different steps; the same SHA then passed when dispatched (38444820216), and so did the stack's top (38444814521).
The e2e spec decofile-api.spec.ts now expects the coded no_repository 404. It moved here from the web half so this PR is green on its own.
apps/api, packages/shared and apps/web typecheck. oxlint, biome and knip are clean.
Locally: storage unit tests 90 pass, the real-daemon e2e 85 pass, the route test 3 pass. Tests that need Postgres/NATS run in CI.
git diff origin/main -- packages/sandbox is empty.
Reapply version (453ebb6b2): PR CI is green, including Checks & Unit Tests 38505127392, E2E 38505127290, Sandbox, Storage Integration, Multi-Pod, Native, SAST and CodeQL. Locally, the hosted and route tests pass (566), and apps/api typechecks.

🤖 Generated with Claude Code

https://claude.ai/code/session_016ZsmzpZ2zahRBoATBb2tBu