feat(opencode): 1h cache TTL on the anthropic system prefix, 5m on the tail

Merged
PI
2mo ago
+44-2

Issue for this PR

Issues are disabled on this fork; motivation below (Slack: #agent-performance cache-TTL thread, 2026-08-24).

Type of change

  • New feature

What does this PR do?

After andytown#25843 froze the volatile clock out of the system prompt, the residual cross-turn cache misses are idle-gap follow-ups: a user replies 5–60 minutes after the last turn and Anthropic's default 5m cache entry has expired, so the ~50k-token prefix re-ingests (~1–1.5s extra TTFT + the 1.25× write premium). Fleet sample: ~78% of follow-up turns land under 5min (already hitting), ~18–25% land in the 5min–1h band.

applyCaching now sets ttl: "1h" on the system-prefix breakpoints only (anthropic provider only); the last-two-message breakpoints keep the 5m default. The 2× 1h-write premium therefore applies just to the rarely-rewritten stable prefix, while per-step tail writes stay at 1.25× — the "~50k static prefix on 1h, conversation tail on 5m" shape from the thread. Anthropic requires longer-TTL entries to precede shorter ones in the request; the system-first layout satisfies that by construction. Back-of-envelope at current fleet volume: converted misses save more in write-premium avoidance than the prefix 1h premium costs (~net −$20–30/day), plus ~1–1.5s TTFT on every 5–60min-gap send.

The AI SDK anthropic provider forwards cacheControl.ttl first-class ("5m" | "1h" in its schema) — verified in the installed package, not assumed.

How did you verify your code works?

New test: system messages carry { type: "ephemeral", ttl: "1h" }, tail messages keep { type: "ephemeral" }, and system precedes tail (the TTL-ordering requirement). Updated the existing non-gateway anthropic cache-control test for the intended new shape — all other provider families' options are byte-identical (full-parity spread). transform.test.ts 291/291, bun run typecheck clean, bun run lint 0 errors. Post-release, the wire effect is visible in Anthropic usage cache_creation TTL breakdown and as turn-initial cache.read>0 in the 5–60min gap band.

Checklist

  • I have tested my changes locally
  • I have not included unrelated changes in this PR

🤖 Generated with Claude Code

Summary by cubic

Splits Anthropic cache TTLs: system prefix now uses 1h, conversation tail stays 5m, reducing cache misses on 5–60 minute follow-ups and improving TTFT. Previously all Anthropic cache entries used 5m TTL.

  • Applies only to Anthropic models in packages/opencode/src/provider/transform.ts: system prefix gets { type: "ephemeral", ttl: "1h" }; tail keeps { type: "ephemeral" }. Other providers are unchanged.
  • Satisfies Anthropic’s ordering rule (longer TTLs first) via the system-first layout.
  • Tests in packages/opencode/test/provider/transform.test.ts cover the TTL split and ordering; existing gateway test updated for the 1h system prefix.
  • Expected impact: fewer prefix re-ingests, ~1–1.5s faster TTFT on idle-gap turns; 2x write premium applies only to the stable prefix.