feat(pages): usercontent origin — Cloudflare Worker + R2, view tickets, server thumbnails (spike)

Merged
SA
2mo ago
+11023-1671

Links

Summary

Spike, end to end, of serving pages from their own origin. A page lives at https://<pageId>.pages.supersetusercontent.com, served by a Cloudflare Worker that reads R2 and never calls the API at request time. Pages are still behind FEATURE_FLAGS.PAGES, so this replaces the interim serving paths outright instead of running beside them.

  1. apps/usercontent — Hono Worker: per-page manifest from R2, HMAC tickets for non-public pages, one CSP for every viewer, comment-runtime injection, thumbnails.
  2. Storage → R2 — pages write through packages/trpc/src/lib/r2.ts; keys mirror URLs. The dead chat-attachment chain (base64 through tRPC in, bytes streamed back through the API out, zero callers anywhere) is scrubbed; chat attachments return on the files layer with the chat UI. Avatars and org logos stay on Vercel Blob until the public bucket lands.
  3. Server-side thumbnails — publish (and setSharedVersion) → QStash → Cloudflare Browser Rendering screenshot of the ticketed page → R2.
  4. Viewers frame the page's URL — web and desktop load viewUrl in a sandboxed iframe with allow-same-origin; srcdoc, superset-page://, the desktop page.content / page.thumbnail routers and the iframe thumbnail cache are deleted.

Also: @superset/shared/usercontent and @superset/shared/page-comments-runtime (the contracts both sides import), a deploy-usercontent job, env plumbing, and the page skill's sandbox notes rewritten for the new model.

Not deployable until P0 below is done — the code is complete and verified locally; the zone, bucket and tokens are not.

Why / Context

Today the same published bytes run three different ways: web frames them as srcdoc under the web app's CSP (opaque origin — every storage API throws), desktop serves them over superset-page:// under a different CSP, and the grid rendered twenty live iframes for thumbnails. All page blobs share one public origin on Vercel Blob. The design settled the fix: a dedicated cookieless domain, one subdomain per page, so a page's script can never reach anything of ours and can safely get a real origin of its own — the githubusercontent.com pattern.

Storage follows serving: the Worker is on Cloudflare, so bytes go to R2 (in-network binding, no credentials, zero egress). "Vercel Blob's CDN" was never a differentiator — Blob is R2 plus Cloudflare's edge underneath.

How It Works

Serving

<pageId>.pages.supersetusercontent.com/                             → shared version, else latest    (no-cache)
<pageId>.pages.supersetusercontent.com/~<ticket>/                   → same, private                  (private, no-store)
<pageId>.pages.supersetusercontent.com/versions/<n>/                → pinned version                 (immutable when public)
<pageId>.pages.supersetusercontent.com/versions/<n>/~<ticket>/      → same, private — relative references inherit the ticket
<pageId>.pages.supersetusercontent.com/versions/<n>/thumbnail.jpg   → ?ticket= when private          (private, max-age=86400)
<pageId>.pages.supersetusercontent.com/_superset/runtime.js

/versions/<n> without the slash redirects to the slash form so relative references inside a version resolve against it (page assets come next). The zone apex and pages. itself redirect to the app.

The Worker reads pages/<pageId>/manifest.json from R2 — {pageId, slug, visibility, sharedVersion, latestVersion, versions: {n: {key, contentType}}} — written by the API on publish, setVisibility, setSharedVersion and delete. No API dependency at request time: an API outage never takes a page down, and flipping visibility is the kill switch. The page id is the host label because it is the one thing never renamed; the slug is only used for the sign-in redirect.

Tickets

everyone serves bare. org / just_me carry an HMAC-SHA256 ticket over {kind: "page", pageId, version?, exp} (WebCrypto; a previous secret is honoured during rotation). Documents carry it as a path segment (/versions/3/~<ticket>/) so relative references inherit it — the door directory publish needs, with no cookie, which Safari would block in our third-party iframe (and which a live check found Claude's artifacts don't use either). Thumbnails keep ?ticket=. Expiry is rounded to a window — an hour for the served alias, a day for a pinned version — so identical claims give identical URLs within the window and browser caches hit; a version-bound thumbnail ticket gets the day. Minted after assertPageReadable on every list/get/pull. A missing or invalid ticket 302s to app.superset.sh/page/<slug>, where the session lives. The ticket carries no identity — it says "this page may be read until exp", nothing more.

One policy for every viewer

default-src 'none'; script-src 'self' 'unsafe-inline'; style-src 'unsafe-inline'; img-src data: blob: https:; media-src data: blob: https:; font-src data: https:; worker-src blob:; form-action 'none'; base-uri 'none'; frame-ancestors https://app.superset.sh file: plus nosniff, Referrer-Policy: no-referrer, X-Robots-Tag: noindex, Origin-Agent-Cluster: ?1 (origin-keyed process isolation while the PSL entry for pages.supersetusercontent.com propagates), and Superset-Storage-Key naming the R2 key — the cache key a client should use, never the URL. list and pull return the same value as storageKey. Network from script is closed; origin isolation, not this header, is the security boundary.

Comment runtime

It only ever needed to run inside the page's document — it never reads content. The Worker injects <script src="/_superset/runtime.js"> before </body> and serves the script same-origin, so it can evolve without republishing anything. The host checks event.origin against the page's origin. #6932's scroll restore rides along in the same script.

Publish

The version number is reserved before the bytes move so the key can name it: pages/<pageId>/versions/<n>/index.html (a new page's id is minted up front). The row is then inserted under that number; a concurrent publish collides on the unique (page, version) index — or finds the target page created by someone else in the meantime — and retries under the next number, deleting the orphaned object. Then the manifest, then the thumbnail job.

Thumbnails

publishPage and setSharedVersion queue {pageId, version} to QStash (dedup id per version). /api/pages/jobs/thumbnail skips if the version is no longer the one served, skips if versions/<n>/thumbnail.jpg already exists (a re-pin is free), mints a 5-minute version-bound ticket, and asks Browser Rendering for a 1280×880 JPEG of the ticketed URL — the same bytes, headers and policy a reader gets. No DB column: the grid <img> falls back to a placeholder on 404.

Storage layout (superset-private)

pages/<pageId>/manifest.json
pages/<pageId>/versions/<n>/index.html      (key stored on page_versions.blob_pathname)
pages/<pageId>/versions/<n>/thumbnail.jpg

Read a key, know the URL, and back; a page is one prefix to list, copy or delete. The public bucket (superset-public, avatars/logos/icons on static.) is CDN-only with no Worker, so it is not bound here — it arrives with the code that writes to it.

Key decisions

DecisionChoiceRationale
Where the serving layer runsCloudflare WorkerThe design already ruled it; superset.sh zone, wrangler and Sentry are already on Cloudflare via relay2; wildcard routes are one line; R2 binding needs no credentials
Host labelPage id, under pages.Never renamed, never user-facing; pages. keeps the top level free for media. and static.
StorageR2, keys mirror URLsCo-located with the Worker; zero egress; no content-addressing, so a key reads as its URL
Manifest in R2 vs. Worker → API lookupManifestServing survives an API outage; four write sites keep it current; idempotent so the thumbnail job repairs a failed write
Ticket formatHMAC over readable claimsVerifiable with the shared secret alone; no JWKS fetch, no identity in the URL; kind keeps a page ticket from opening anything else
Thumbnails: eager on publish vs. lazy on viewEager, async, plus on re-pinThe grid only ever shows sharedVersion ?? latest; existence is the state, so re-pinning a captured version costs nothing
Who runs the browserCloudflare Browser RenderingUntrusted HTML never executes with the API's network position; hosted isolation
Content CSPNetwork closed, remote images/fonts allowedMatches desktop's existing PAGE_CSP and the spec's proposal; a per-page loosen is a later toggle
Desktop embedding<iframe> + frame-ancestors file:Packaged renderer loads via file://; a frame still needs a ticket, so this admits embedding, not reading
allow-same-originOnPer-page origins make it safe; this is SUPER-1999's option B
Bucketssuperset-private + superset-publicThe public bucket earns a custom domain and CDN with no Worker; only possible as its own bucket
SchemaNo changeblob_pathname holds the R2 key; thumbnails are derived, not tracked

What has to happen before deploy (P0)

Tracked as SUPER-2080. Previews now use superset-private-dev with USERCONTENT_TOKEN_SECRET_DEV / vars.USERCONTENT_URL_DEV — create those alongside the production values.

  1. Move supersetusercontent.com from Vercel nameservers to the Superset Cloudflare account; proxied *.pages, pages and @ records; Advanced Certificate Manager for the second-level wildcard.
  2. R2 buckets superset-private and superset-private-dev; an S3 API token. (superset-public comes with the attachments layer.)
  3. A Browser Rendering token; Workers Paid for its rate limits.
  4. Env on the API/web Vercel projects and GitHub secrets/vars: CLOUDFLARE_ACCOUNT_ID, R2_ACCESS_KEY_ID, R2_SECRET_ACCESS_KEY, R2_PRIVATE_BUCKET, USERCONTENT_URL, USERCONTENT_TOKEN_SECRET (also wrangler secret put on the Worker), CLOUDFLARE_BROWSER_RENDERING_TOKEN.
  5. bun --env-file=.env packages/trpc/scripts/migrate-pages-to-r2.ts once, before cutover (copies existing versions to the registry keys, repoints the rows, writes manifests; idempotent).

Manual QA Checklist

Not run against a deployed origin — blocked on P0. The Worker itself was exercised locally (see Testing).

Serving

  • org page: opening in web/desktop loads; the raw URL without ?ticket= redirects to the app viewer
  • everyone page: raw URL loads with no ticket
  • Republish → / serves the new version; pinned /versions/<n>/ still serves the old one
  • Flip a page to just_me → a teammate's open frame stops loading on reload
  • Delete a page → 404 within a request

Page behaviour

  • localStorage persists across reload and across versions of the same page
  • fetch() and a remote <script src> are blocked; a remote image loads
  • Comments: hover, pick, pins and rects work on web and desktop; scroll position restores on reopen

Thumbnails

  • Publish → thumbnail appears in the grid within ~10 s; placeholder before that
  • Republish twice quickly → only the latest version is rendered
  • Pin an older version that was never captured → its thumbnail appears

Desktop

  • Packaged build frames a page (frame-ancestors file:) — unverified assumption, see limitations

CLI / MCP

  • superset pages pull still round-trips (presigned URL)

Testing

  • typecheck — @superset/shared, trpc, ui, content, api, web, desktop
  • biome check --write on touched files
  • bun test: packages/trpc/src/router/page 50, packages/ui PageComments 36 — pass. publish.integration.ts (not in CI) needs a database whose schema is current; the local .env Neon branch is behind main, so it fails in beforeAll before any page code runs.
  • Ticket verifier and URL builders: 15 unit tests (round-trip, expiry boundary, rotation list, tampering, wrong kind, malformed input; path forms and relative-reference inheritance).
  • Worker, locally against wrangler's simulated R2 (wrangler dev --env dev, manifests and versions seeded at the registry keys, tickets minted with the shared signer): 22 cases — public / no-cache with the runtime injected, /versions/1/ immutable, /versions/1 → 301 to /versions/1/ (query preserved), unknown / non-numeric version → 404; private: no ticket → 302 to the app, page ticket → 200 on the served version (private, no-store), version-bound ticket opens its version and is refused for another, expired / other-page / legacy ?t= → 302; thumbnail 200 image/jpeg with a ticket, 404 without, 404 when absent; slug-style host label → 404; apex /health 200, apex / → app; /_superset/runtime.js 200; CSP, nosniff, no-referrer, noindex headers present.

Bot review triage (CodeRabbit, 14 inline comments)

Fixed on this PR: the publish upload/reservation race (upload now inside the version-reserving transaction; unsafe delete-cleanup removed); restricted-thumbnail browser caching now bounded by the ticket's remaining lifetime; page delete fails closed (manifest removed first) and manifest writes retry; desktop thumbnail placeholder retries after a minute; deploy-usercontent job restricted to contents: read.

Dismissed, with reasons: img-src/media-src https: beacon exfiltration is the documented remote-media tradeoff (a page can only exfiltrate its own page-scoped ticket; the skill documents the privacy cost); preview content-origin alignment is moot (preview deploys are unsupported by decision); the chat-attachments migration comment targets a route and table this PR deletes; the anonymous-user fallback and SKILL hostname comments were addressed in earlier commits. Durable manifest reconciliation (outbox) beyond the added retry stays a recorded follow-up.

Known Limitations

  • The chat_attachments table (migration 0046) outlives its removed code Dropped here after all (migration 0100) while pages are dark, together with a true-rename of page_versions.blob_pathname → storage_key (Blob-era name; old rows keep their Blob pathnames through the rename for migrate-pages-to-r2.ts) and the stranded serverError.chat.failedToRecordChatAttachment catalog key. The stacked files-layer PR regenerates its own migration 0100 as 0101 on rebase.

  • Not verified deployed. Real Browser Rendering capture, the R2 S3 write path, and QStash delivery are exercised only by reading the docs and the shape of the sibling code.

  • frame-ancestors file: for the packaged desktop app is an assumption — verify in a build before release; the fallback is a <webview> with an IPC transport for the comment bridge.

  • Tickets expire after 1 h; a frame left open past that shows the sign-in redirect on reload (the viewer mints a fresh one).

  • wrangler dev with routes rewrites every host to the zone, so local dev uses the routeless --env dev and curl --resolve '<pageId>.pages.usercontent.localhost:8787:127.0.0.1'; Chromium resolves *.localhost on its own.

  • No new unit tests (the ticket verifier is the one place worth some).

Follow-ups (the doc's order of work)

  • Media route on the Worker and the page door: assets at manifest paths, partitioned cookie, style-src 'self'; directory publish in the CLI and MCP.
  • Attachments layer: files + attachments tables, presigned direct-to-R2 PUT, serve-time content policy; chat composer and avatars switched over; public bucket + static. custom domain; @vercel/blob removed.
  • Mobile page viewer as a ticketed WebView URL.

Rollout / Rollback

  • Rollout: P0 → migration script → deploy Worker (deploy-usercontent) → deploy API + web. Pages stay behind the flag throughout.
  • Rollback: revert the PR. Objects in R2 and the Worker are inert if nothing points at them; Vercel Blob still holds every existing version until the bucket is emptied by hand.

Files changed

New

  • apps/usercontent/ — the Worker (src/index.ts, src/types.ts, wrangler.jsonc, .dev.vars.example)
  • packages/shared/src/usercontent/ — keys, manifest, ticket signing/verification, URL builders, CSP, script injection
  • packages/shared/src/page-comments-runtime.ts — moved from packages/ui
  • packages/trpc/src/lib/r2.ts — put/get/head/delete/presign over aws4fetch
  • packages/trpc/src/router/page/storage.ts — manifest writer, ticket minting, object cleanup
  • packages/trpc/src/router/page/thumbnail.ts — enqueue + generate
  • apps/api/src/app/api/pages/jobs/thumbnail/route.ts — QStash-signed job
  • packages/trpc/scripts/migrate-pages-to-r2.ts — one-off backfill

Removed

  • apps/desktop/src/main/lib/pageContent/, apps/desktop/src/main/lib/pageThumbnails/, apps/desktop/src/lib/trpc/routers/page/, the superset-page:// and superset-thumb:// schemes
  • .../PageThumbnail/utils/pageThumbnailCache/ (iframe thumbnails)
  • apps/web/src/app/page/[slug]/utils/getPageContent/
  • packages/ui/.../utils/commentRuntime/ (moved to shared)

Summary by CodeRabbit

  • New Features

    • Pages are now served from dedicated user-content origins with improved isolation and access controls.
    • Page previews use server-generated thumbnails and load more efficiently across web and desktop.
    • Added secure, expiring links for private page views and downloads.
    • Page comments continue working when viewing published pages.
    • Chat attachments and page content now use more reliable managed storage.

  • Bug Fixes

    • Improved handling of missing or unavailable attachments and page content.
    • Added validation for thumbnail jobs and protected page requests.