refactor(usage): handle user credits as a counter kind on the shared quota path

Merged
ET
1mo ago
+4428-6666

Stacked on #27944. The user allowance becomes a third counter kind on main's quota path, built, warmed, debited and admitted like the workspace allowance, instead of running as a parallel engine next to it. The decided behaviors do not change: user credits pay first with an exact split, spill or STOP, several windows, the userCreditsUsedMicro column, and the chat stop and notice.

Size

  • Against the #27944 head: 118 files, +4,428 / -6,666. That is -1,606 net source, -554 tests, -78 generated.
  • Whole feature against main:
#27944with this PR
usage-limit source+1,511+483
AI source+502+104

Changes

Usage-limit

  • UsageLimitQuotaService owns the quota path again, as on main. The counter builder, counter store and debit services are deleted.

  • WARM_AND_CONSUME_COUNTERS_SCRIPT debits every counter. Per-key split tags (user, workspaceCredits) let it compute the user share atomically.

    • Gone: the user-first script, its argument builder and reply parser, the three-outcome result, the settle step and the second limit pass.
    • A debit makes one Redis call again.
  • filterExhaustedCountersByPayer decides who pays and what blocks, for both debit and admission. UsageQuotaAdmissionService is back to main's four methods and returns only a refusal.

  • User windows are built like any quota default counter (kind userAllowance). The fallback is read together with the workspace allowance counter.

  • Every counter warms from the shared consumption rows. computeRowConsumed counts:

    • userCreditsUsedMicro for USER_CREDIT;
    • creditsUsedMicro - userCreditsUsedMicro for workspace CREDIT limits.

    The limit-form preview uses the same rule. Migration 014 rebuilds consumption_by_scope and billing_by_workspace_period with the user sum, so getUserCreditsUsedMicroForScope is gone.

Billing and usage

  • Billing-period workspace and user credits come from one query. The CreditsUsage type is gone.
  • Analytics select total and user credits directly, instead of subtracting in SQL and adding back in TS. The time series is back to main's shape.
  • The recorded user share is clamped once, in the chat turn. RecordUsageInput carries an optional userCreditsUsedMicro, and the ClickHouse envelope always writes it.

AI

  • The step debit returns { stopReason, userCreditsUsedMicro, userAllowanceExhaustedUntil }.
  • The chat sends user-credits-exhausted from the debit that crosses a window, whether a step or the end-of-turn web search.
  • One stopReason goes to the job and the turn classifier, and the turn keeps a running user credits total.
  • No credits check at job start. As for the workspace allowance, the preflight and the per-step debit decide.
  • Removed:
    • SendChatMessageResult.hasWorkspaceCreditsLeft
    • AiChatUsage.userCredits and its reader
    • AiExceptionCode.USER_CREDITS_EXHAUSTED
    • createTurnUsageRecorder
    • the turn-start snapshot and the period-end getters
  • Reverted unrelated hunks: the streaming reformat, and agentId on agent web searches.

Behavior changes against #27944

  • Degraded paths follow main:
    • If ClickHouse cannot warm a user window, the charge is admitted. Warm workspace counters still take the full amount, and the recorded user share is 0.
    • A script error admits the charge.
    • The admitted-on-failure metric counts once per charge.
  • In STOP mode, a queued message, retry or setup turn that starts after the user ran out runs one step, then stops. Main does the same for the workspace allowance. New sends are still refused by the preflight.
  • Errored or interrupted chat turns again write no usage row, as on main. "Record errored and interrupted chat turns" (f33d1527984) should ship as its own PR.
  • In STOP mode the notice goes out at the step that stops the turn. A crossing during title generation or tool repair sends no notice.
  • The [AI_CHAT_NO_TEXT] log prints stopReason= instead of outOfCredits=.
  • The front branch must stop reading hasWorkspaceCreditsLeft and AiChatUsage.userCredits.

Deploy

  • Run ClickHouse migration 014 before deploying: every usage insert now writes userCreditsUsedMicro.
  • 014 rebuilds both projections without MATERIALIZE, as 012 did. Parts written before it fall back to raw reads.

Open question

In STOP mode, the web searches of an agent run started by a user are charged to that user's credits even when they are used up. #27944 does the same, because the matching quota decides who pays. Changing it needs a product call.

Not done

userCreditsFallback stays in the front user fragment. BillingCustomer.userCreditsFallback is non-null, and the front's current-workspace type breaks without the selection.

Testing

  • Server typecheck is clean.
  • Unit specs pass in usage-limit, usage, billing, billing-webhook, admin-panel, twenty-config, logic-function, metadata-modules/ai, emailing and workflow-executor.
  • Lint and format are clean on the changed files.
  • The extended Lua script ran against a local Redis on 12 scenarios (35 checks):
    • plain limits
    • user pays all
    • exact split at the crossing
    • full window
    • STOP overshoot
    • two windows
    • cold user key deferred, then applied once on retry
    • cold workspace key
    • no workspace key
    • stock release and consume unchanged
  • New integration cases in quota-consumption: user pays first, exact split, STOP, and a failed warm. They need billing enabled and were not run locally.
  • Front AI util tests pass. The SDK and front metadata types were regenerated offline.