fix(credits): meter LLM credits when a turn fails after a side-effect

SW
4mo ago
+70-26

Tracked by chat#1885 (billing-integrity row; sibling to api#782/#783 — recommend splitting into a dedicated billing-integrity tracker).

Problem

runAgentWorkflow charged the account + wrote the usage_events audit row only at app/lib/workflows/runAgentWorkflow.ts L119, gated on if (result.responseMessage) after runAgentStep returned — inside a try whose only companion is a finally (no catch).

The customer email is a send_email tool call executed inside runAgentStep (mid-turn). So a turn that emails the customer and then throws — or returns without a responseMessage — skips billing entirely: email out, wallet not debited, no usage_events row.

getCreditUsage/handleChatCredits can't be the cause: an unknown/zero-priced model still writes a ≥1¢ row with model_id (Math.max(1, …)). Zero rows ⇒ handleChatCredits was never called.

Evidence (2026-07-23)

A scheduled briefing for account fade8fd2-… on the new moonshotai/kimi-k3 default emailed "Brauxelion Weekly" at 20:54 but wrote 0 usage_events rows with a model_id. The earlier anthropic/claude-sonnet-5 run wrote 1 (106¢, 2.63M in / 31.8k out). Both runs' model-independent scrape/tool charges (235¢, model_id=null) were identical — only the LLM line vanished. With Kimi now the default for every headless briefing, this risks systemic email-but-not-bill with no audit trail.

Fix

Extract the charge into an idempotent chargeTurn() and call it (a) on the normal path and (b) as a finally backstop. A turn that produced a customer-facing side-effect now always lands a wallet debit + audit row; ZERO_USAGE with no gateway cost floors to the 1¢ minimum, so a failed turn still writes a model_id row. chargeTurn guards on a charged flag so the normal + user-abort paths bill exactly once. The original error still propagates after the backstop runs.

Behavior

Turn outcomeBeforeAfter
Normal finish (responseMessage)1 charge (real usage)1 charge (real usage) — unchanged
User-abort (responseMessage)1 charge1 charge — unchanged
Returns no responseMessage0 charges ❌1 charge (1¢ floor + model_id row) ✅
Throws after send_email fired0 charges ❌1 charge (1¢ floor + model_id row) ✅; error still propagates

Tests

TDD red→green. Flipped the two tests that codified the gap ("does NOT bill when no responseMessage" / "…when it throws") to assert the turn is now billed with the 1¢ floor.

  • pnpm exec vitest run app/lib/workflows/__tests__/runAgentWorkflow.test.ts → 20 passed
  • pnpm exec vitest run app/lib/workflows lib/credits → 129 passed
  • pnpm exec eslint on both changed files → clean

No API contract change → no docs PR (a billing side-effect, like api#782). Note: tsc --noEmit has 203 pre-existing errors on main (trigger/tasks test fixtures); this PR adds none, and runAgentWorkflow.ts is type-clean.

Follow-up (out of scope)

Confirm why Kimi returned no responseMessage in this workflow (Vercel workflow run logs: finishReason / thrown error) — that's a model-integration question separate from this billing-integrity fix.

🤖 Generated with Claude Code


Summary by cubic

Ensures LLM credits are charged when a turn fails or returns no response after a side-effect, preventing email-but-not-billed gaps. Billing now always records a usage_events row with a 1¢ floor.

  • Bug Fixes
    • Added idempotent chargeTurn() and invoke it on success and in finally.
    • Bills user-stop and failure paths; uses ZERO_USAGE → 1¢ minimum; writes a model_id row.
    • Guards to charge exactly once; original errors still propagate.
    • Updated tests to assert billing on no-response and thrown-turn cases; no API changes.