feat: move billing to Autumn and meter cloud boxes on balance locks, backward compatible

Merged
HA
1mo ago
+20548-1005

Moves Superset's billing state to Autumn, which drives our existing Stripe account, and meters cloud workspace boxes on Autumn balance locks with spend-cap enforcement. Released desktop and web builds keep working unchanged.

Execution plan with decisions and status: plans/20261013-1755-autumn-billing-and-box-metering.md.

Summary

  • Box metering on rolling locks. Every box start or wake checks Autumn and takes a short balance lock before the box resumes; the reconciler finalizes the memory GB-minutes used and takes the next lock every 10 minutes; stopping, archiving or deleting settles the rest. A free organization, or one at its spend cap, is refused with a clear message, and a running box whose next lock is refused is stopped.
  • Plan state from Autumn (with BILLING_PROVIDER=autumn): plan gates and billing.activePlan read Autumn behind a short cache; tiers come from the pro_access / enterprise_access features, so a new plan needs no code.
  • Checkout, plan changes and usage go through /api/auth/autumn/* routes with our owner/admin rules. Released clients keep calling the Stripe plugin's /subscription/* routes, now served from Autumn by a clearly marked legacy shim.
  • Dunning notices finally run: the payment-failed and renewal emails, Slack notices and payment analytics were registered where the Stripe plugin never called them; they now send in both modes.

How It Works

Metering. One row per box start in cloud_workspace_boots (migration 0146): a boot id minted before the box resumes, the Vercel session it became, the organization (no foreign key, so usage outlives deletes), memory, the current lock and how far it is finalized. Usage is memory GB-minutes (minutes the box was up times its memory in GB); prices live only in Autumn.

  • Create: assertMayBoot refuses before the workspace row is written.
  • Provision and wake: authorizeBoot takes lock bootId:0 before the box resumes; the boot is tied to its Vercel session afterwards.
  • Reconcile (QStash, every 10 minutes, Sentry cron monitor): queues settleBoot for every open boot and adopts boxes already running without one. settleBoot finalizes the minutes since the last settle, then renews the lock or closes the boot and records active CPU as an unbilled event. Settles are serialized by a lease column; a lock that cannot be finalized falls back to an idempotent track keyed by the lock id.
  • Refusals: FORBIDDEN with serverError.cloudWorkspace.cloudNeedsPaidPlan or serverError.cloudWorkspace.usageLimitReached. Desktop shows them on create (with "Open billing") and as a banner when a wake is refused; mobile shows a "can't start" screen; automation runs record cloud_usage_limit_reached. Environment builds are not billed.

Plan state. organizationBilling(orgId) reads the customer's access features, subscription and invoices from Autumn behind a per-instance cache and Redis, serves a last-known copy if Autumn errors, and fails open only without one. activePlan keeps every field released clients read (plan, status in Stripe's vocabulary, periods, interval, lapsed) and adds paymentState and billingProvider. The subscriptions table stays as a projection for SQL readers (session plan, sync policy, automations, admin analytics): Autumn does not expose Stripe subscription ids, which seat sync, Slack jobs and purge need.

Routes. /api/auth/autumn/{attach, previewAttach, updateSubscription, previewUpdateSubscription, openCustomerPortal, getOrCreateCustomer, listPlans, aggregateEvents} come from our own better-auth plugin rather than autumn-js/better-auth behind a hook: a before hook runs before the bearer plugin builds a desktop's session. Each endpoint runs after the session middleware, applies owner/admin rules (card and billing address owner-only), refuses enterprise plans, and whitelists its body, so the customer, seats and checkout settings come from the server.

Layout. packages/auth/src/lib/billing/autumn/ is permanent; legacy/ holds the Stripe-routes shim, its Stripe fallback and plan ids, each @deprecated with a removal TODO, and a README listing when each goes.

Backwards compatibility

  • BILLING_PROVIDER unset: the Stripe plugin runs as before (its onEvent notices now actually send). Box metering and the boot check run in both modes.
  • BILLING_PROVIDER=autumn: released clients call the same /subscription/upgrade|cancel|restore|success routes with the same paths, body schemas, middleware, responses and error codes. Subscribers Autumn does not manage yet switch interval and restore through their existing Stripe subscription, so nothing double-bills before the import finishes.
  • No tRPC input changes; activePlan only gains optional fields.

Testing

  • bun test: packages/auth 52, packages/trpc 582, apps/api 230, packages/db 21, packages/shared 1313, packages/i18n 165, all pass. New: settle paths (renew, close, cap refusal, finalize fallback, lease), refusal mapping, GB-minute unit, Autumn-to-activePlan mapping per status, route role and checkout policy, the shim's routes and middleware (including a real better-auth instance refusing a foreign callbackURL), the Stripe fallback against a stubbed client, automation error mapping.
  • Desktop: 7980 pass; 36 failures are environmental real-git, shell and terminal-transport tests, not in files this PR touches (the same terminal-transport tests fail on the base commit).
  • Typecheck matches the base in every package; repo-wide biome check and sherif clean; check:i18n compiles every locale, with all new strings translated.
  • Not run against live Autumn, Stripe or Vercel.

Deployment / Rollout

  1. Before merge: gh secret set AUTUMN_SECRET_KEY and AUTUMN_WEBHOOK_SECRET for Production and Preview (Autumn's sandbox key on Preview). They are required; a deploy without them fails at boot.
  2. In Autumn: connect Stripe; create the features box_gb_minutes, active_cpu_minutes, seats, pro_access, enterprise_access; plans hobby (auto-enabled, no box_gb_minutes), pro and pro_annual (reusing the existing Stripe price ids) and enterprise; a webhook to /api/integrations/autumn/webhook for billing.updated.
  3. In the Upstash console, add a QStash schedule every 10 minutes for /api/cloud-workspaces/jobs/reconcile-usage.
  4. Confirm in the Autumn sandbox: a retried and an expired finalize, a fractional overrideValue, track on an unpriced feature.
  5. Run the "Import Subscriptions into Autumn" workflow (dry run, then apply); flip only when it reports 0 skipped and 0 drifted.
  6. Set BILLING_PROVIDER=autumn on Preview, verify, then Production. The flip is one-way once verified.

Rollback

  • Before the flip: unset BILLING_PROVIDER (nothing changes). Pausing the QStash schedule stops metering renewals; open locks expire and release their holds.
  • After the flip, Stripe mode is an emergency fallback only; subscriptions created by Autumn are not linked back to the Stripe plugin.

Known Limitations

  • Autumn lock behaviours above are unverified until step 4.
  • Boxes still run until Vercel's session timeout; every minute is billed until our own idle stop lands.
  • Not covered by unit tests: the plan-state cache tiers and the moved Stripe dunning handler.