fix: [AI-6622] enable cache_control trigger for altimate-backend provider

Merged
SA
5mo ago
+1-0

Summary

Add model.providerID === "altimate-backend" to the cache-control gating condition in ProviderTransform.message() so messages routed through the Altimate Gateway get cache_control: ephemeral markers stamped on the last content block.

Why this matters

The Altimate Gateway uses litellm with provider fallback: Azure GPT-5.5 (primary) → Anthropic Sonnet 4.6 (fallback). When Azure errors and litellm falls back to Anthropic, Sonnet honors the cache_control markers and serves cached input at a discount.

Without this fix, fallback Sonnet calls would not cache, paying the full input rate.

For the Azure-primary path: Azure ignores cache_control markers (it uses automatic prefix-based caching), so stamping them is a no-op there.

Empirical context

Full RCA: ~/Documents/altimate-deliverables/AI-6622-{repro-summary,investigation-journey}.md.

We empirically proved the gateway's Azure path caches at 97-99% (response field usage.input_tokens_details.cached_tokens); the visible-cache-as-zero symptom that originated AI-6622 was a backend observability gap, not a cache failure. Companion backend fix: AltimateAI/altimate-backend#... extracts and surfaces the cached_tokens field so altimate-code can render real cache rate + cost.

Test plan

  • transform.ts:284-296 is the only modified location; single line addition
  • Build the CLI, run altimate-code with the Altimate Gateway model selected, force an Azure failure so litellm falls back to Anthropic, confirm response includes cache_read_input_tokens > 0 on warm turns

🤖 Generated with Claude Code

Summary by cubic

Enable cache_control markers for the altimate-backend provider so Altimate Gateway requests mark the last content block as ephemeral. This satisfies AI-6622 by allowing Anthropic fallback to cache input and reduce cost, while Azure primary remains unaffected.

  • Bug Fixes
    • Add model.providerID === "altimate-backend" to the cache-control gating in ProviderTransform.message().

Summary by CodeRabbit

  • Performance
    • Enabled caching optimization for the altimate-backend provider to enhance application performance.

Review Change Stack