Use the Pydantic AI Gateway's reported cost for `RequestUsage.cost`

Merged
AD
5d ago
+402-15
  • RequestUsage.cost uses the cost the Pydantic AI Gateway reports in the response body (usage.logfire_ai_gateway.cost, falling back to the deprecated usage.pydantic_ai_gateway.cost_estimate) instead of the genai-prices estimate, for OpenAI Chat/Responses, Groq and Anthropic, streamed and not. A reported 0 (response-cache hit) counts.
  • Google and Bedrock SDKs drop that body field, so on non-streamed requests they read the logfire-ai-gateway-cost header instead. It holds the same unrounded cost as the body, so it also wins over the genai-prices estimate, and 0USD (response-cache hit) counts. The deprecated, rounded pydantic-ai-gateway-price-estimate header isn't read. Streamed Google and Bedrock responses carry no gateway cost and keep the estimate.
  • InstrumentedEmbeddingModel reports usage.cost when set, matching model telemetry.
  • cost() on responses and embedding results still returns the genai-prices estimate.
  • Docs: one "Request cost" section in docs/gateway.md.
PR decisions

2026-10-24 · Reported cost telemetry precedence · iter -

2026-10-24 · Adopted PR #7337 · iter -

2026-10-24 · Preserve upstream Anthropic package split · iter -

  • Decision: Port Gateway cost carry-through into models/anthropic/_response.py::map_usage and preserve upstream init.py unchanged.
  • Why: Main moved usage mapping in the pure module split; keeping the old module would discard upstream structure.
  • Source: https://github.com/pydantic/pydantic-ai/pull/11165

2026-10-24 · pre-push review 1 of 3 at 241bb3dc67f0ed87e464fbf6175c9e96bae96878 · iter -

2026-10-24 · pre-push review 1 outcome at 241bb3dc67f0ed87e464fbf6175c9e96bae96878 · iter -

2026-10-24 · pre-push review 2 of 3 at 01c4cb85fc3ae0d1411ae25be3c73057664d1e99 · iter -

2026-10-24 · pre-push review 2 outcome at 01c4cb85fc3ae0d1411ae25be3c73057664d1e99 · iter -

2026-10-25 · Additional review selection · iter -

2026-10-25 · Preserve embedding pricing degradation · iter -

  • Decision: Inline reported-cost precedence while preserving the existing non-Gateway estimation error behavior; do not replace it with best_effort_price in this PR.
  • Why: Public target/candidate probes both warn on ValueError; the proposed shared-helper replacement suppresses that warning and changes users outside the Gateway fix.
  • Source: https://github.com/pydantic/pydantic-ai/pull/7337#issuecomment-6392299990

2026-10-26 · Ignore rounded deprecated cost headers · iter -

  • Decision: David chose to ignore pydantic-ai-gateway-price-estimate; retain legacy body and canonical header support, with genai-prices fallback when available.
  • Why: Recorded positive Bedrock cost rounds to zero in the deprecated header; treating that as authoritative can bypass a small client cost limit.
  • Source: https://github.com/pydantic/pydantic-ai/pull/7337#issuecomment-6392299990

2026-10-26 · pre-push review 3 of 3 at 12746fe4916560f6bb9cc6477aa3616d4116f063 · iter -

2026-10-26 · pre-push review 3 outcome at 12746fe4916560f6bb9cc6477aa3616d4116f063 · iter -

2026-10-26 · Gateway header cost is a fallback only · iter -

  • Decision: Exact body cost wins; the rounded header is used "only when the response can't be priced", and a zero "only on a response-cache hit".
  • Why: The gateway writes both cost headers with toFixed(4), so small costs round to 0.0000USD; maintainer (Aditya) ruling, checked by an independent Opus review.
  • Source: https://github.com/pydantic/pydantic-ai/pull/7337

2026-10-26 · pre-push review 1 of 3 at b6f888de0b448e770e7b1eeb561242f00c86ce0b · iter -

2026-10-26 · Deprecated cost header read as a fallback · iter -

  • Decision: Read both header names; the header is used "only when the response can't be priced", and a zero "only on a response-cache hit".
  • Why: Rounding only harms a header that overrides the estimate; as a fallback the deprecated header works on today's deployed gateway.
  • Source: https://github.com/pydantic/pydantic-ai/pull/7337
  • Supersedes: Ignore rounded deprecated cost headers

2026-10-26 · pre-push review 1 outcome at b6f888de0b448e770e7b1eeb561242f00c86ce0b · iter -

2026-10-26 · pre-push review 2 of 3 at 77471aac452099bd00b02f26604d0c030d3be3c0 · iter -

2026-10-26 · pre-push review 2 outcome at 77471aac452099bd00b02f26604d0c030d3be3c0 · iter -

2026-10-26 · pre-push review 3 of 3 at ae7751f98540fa2655af911ea5a509aad81fb780 · iter -

2026-10-26 · pre-push review 3 outcome at ae7751f98540fa2655af911ea5a509aad81fb780 · iter -

2026-10-26 · post-budget lint fix at 2758aeb54316a7e7a8c42e954b34dd14a22c1d87 · iter -

2026-10-26 · post-budget review-thread fix at 08aed6ac38600e44cdb1170247a7a980335654ba · iter -

  • Decision: Applied CI Review suggestion: read Anthropic gateway blocks from model_extra; no fourth pre-push review (budget exhausted).
  • Why: One-line change; verified anthropic construct_type stores the blocks in model_extra; Anthropic recorded, stubbed and streamed tests pass.
  • Source: https://github.com/pydantic/pydantic-ai/pull/7337

2026-10-26 · Gateway cost header wins over the estimate · iter -

  • Decision: logfire-ai-gateway-cost sets RequestUsage.cost like the body cost; no x-cache-status: HIT check for a zero. Merge once the gateway sends the unrounded header in production.
  • Why: The gateway now sends the header unrounded, the same value as the body; it's set on non-streamed responses only.
  • Source: https://github.com/pydantic/pydantic-ai/pull/7337
  • Supersedes: Gateway header cost is a fallback only; Deprecated cost header read as a fallback