fix: stop billing cached Anthropic/Bedrock input tokens as uncached

AN
3d ago
+123-19

Issue for this PR

Closes #1545

Type of change

  • Bug fix
  • New feature
  • Refactor / code improvement
  • Documentation

What does this PR do?

With ai 6 and @ai-sdk/anthropic 3, usage.inputTokens is the inclusive total, and usage.inputTokenDetails splits it into no-cache, cache-read and cache-write tokens. Session.getUsage still assumed Anthropic and Bedrock report uncached input only. So a cached step was charged the whole input at the uncached rate, plus the cache read and write charges.

  • Session.getUsage uses inputTokenDetails when the SDK reports noCacheTokens. Older usage without the split takes the old path.
  • The over-200K price tier now counts uncached input, cache reads and cache writes. Before this change the inclusive input stood in for that total.
  • The learning flows' accountUsage now passes inputTokenDetails through. It used to drop them, so it had the same bug.

Behaviour changes:

BeforeAfter
Anthropic/Bedrock step cost with cache hitsinput billed twice (uncached rate + cache rate)billed once, split by cache type
tokens.input (traces, tokens_input telemetry)inclusive totaluncached only, as documented
tokens.inputTotalinclusive + cache read + cache writeinclusive total
OpenAI, OpenRouter, Gemini, Copilot—unchanged (the split equals the old subtraction)

Who this affects: everyone on Anthropic, Bedrock or Vertex-Anthropic models. Their displayed session cost, trace cost and cost telemetry fall to the real figure, about 1/6 on cache-heavy sessions. Dashboards that track cost over time will show a step down at this release.

How did you verify your code works?

  • Unit tests: new session-getusage cases use the exact v6 usage shape: Anthropic cost on a real step's token counts, Bedrock, a full cache hit, cache write taken from the details, and the OpenAI v6 shape. I added a learning-flow cost test.
    • The new session-getusage cases fail on main, run in a detached worktree at main: 3 of the 15 cases in that file failed.
    • Every suite that calls getUsage/accountUsage passes: 2,085 pass, 0 fail (2,036 pass, 0 fail on the final head across the getUsage suites).
  • Typecheck and markers: bun run typecheck is clean, and script/upstream/analyze.ts --markers --base main --strict passes.
  • Real traces: I recomputed 164 Anthropic Opus agent sessions from their own token counts and list prices. Reported $229.49, real $36.78, a 6.2x ratio. Each step matches the old formula exactly: input × $4 + cacheRead × $0.20 + cacheWrite × $5 + output × $20, with the inclusive input.
  • Review: Codex was out of credits, so a Sonnet agent did the independent review. It reported no P0 or P1 findings. Its one P2, the learning-flow path, is fixed here.
  • Not run: a live session against the provider after the fix.

Screenshots / recordings

Not a UI change.

Checklist

  • I have tested my changes locally
  • I have not included unrelated changes in this PR

🤖 Generated with Claude Code

[!NOTE]
Medium Risk
Changes core token and cost accounting for all Anthropic-family sessions; reported costs and tokens_input telemetry will drop on cache-heavy workloads, while OpenAI-style providers should stay on the equivalent split/subtraction path.

Overview
Fixes over-billing on Anthropic, Bedrock, and Vertex-Anthropic when AI SDK v6 reports inclusive inputTokens plus inputTokenDetails (no-cache / cache-read / cache-write). Session.getUsage had still treated those providers like the old shape where inputTokens was uncached-only, so cache-heavy steps were charged at the full uncached rate and cache read/write — often ~6× the real cost.

Session.getUsage now keys off noCacheTokens in inputTokenDetails when present: uncached input, cache read/write, and cost use that split; legacy usage without details keeps the previous subtraction/metadata paths. The over-200K tier threshold now includes cache-write tokens. Learning accountUsage forwards inputTokenDetails into session accounting (and extends GenerateUsage with noCacheTokens). Telemetry comments were updated to match. New unit tests cover the v6 usage shape and learning cost parity.

Reviewed by Cursor Bugbot for commit 3267982c749e5eaf879b496eebcd8b4f8d8c954b. Bugbot is set up for automated code reviews on this repo. Configure here.

Summary by cubic

Fixes over-billing for cached Anthropic, Bedrock, and Vertex-Anthropic sessions that charged cached input tokens at the uncached rate on top of cache read/write charges. Closes #1545.

With ai 6 and @ai-sdk/anthropic 3, usage.inputTokens is the inclusive total and usage.inputTokenDetails splits it into uncached, cache-read, and cache-write counts. Session.getUsage still assumed Anthropic and Bedrock reported only uncached input, so a cached step was billed for the full input twice — about 6x the real cost on cache-heavy sessions.

  • Session.getUsage now uses inputTokenDetails when present; usage without the split keeps the legacy path.
  • Learning-flow accountUsage passes inputTokenDetails through instead of dropping them.
  • tokens.input now reports uncached tokens only; tokens.inputTotal reports the inclusive total.
  • The over-200K price tier now counts cache writes toward the threshold.
  • Step cost for Anthropic-family models drops to the real figure; OpenAI, OpenRouter, Gemini, and Copilot are unchanged.
  • Cost-over-time dashboards will show a downward step at this release.

Summary by CodeRabbit

  • Bug Fixes
    • Improved token usage and cost reporting when providers supply cached and uncached input details.
    • Added support for newer usage data formats, with fallbacks when detailed cache information is unavailable.
    • Updated high-volume pricing calculations to include cache-write tokens when determining whether input exceeds 200,000 tokens.