feat(opencode): configurable prompt cache key for key-routed providers

Merged
PI
2mo ago
+65-5

Issue for this PR

Issues are disabled on this fork; motivation below (terra/OpenAI caching experiment, follow-up to #30's Anthropic-side work).

Type of change

  • New feature

What does this PR do?

OpenAI routes prompt caching by key: cached KV state lives on individual machines, and traffic above ~15 req/min for a given prefix + prompt_cache_key overflows to machines that don't hold the entry (their docs). Upstream opencode sends promptCacheKey = sessionID, which keeps each session sticky but means a user's other sessions and forks never share a warm cache — every new session re-warms the full shared prefix (system prompt + tools) from scratch, and OpenAI's own guidance is to key per user/agent pair with session IDs kept out of the key.

This adds a promptCacheKey provider option (string, next to the existing setCacheKey) that overrides the per-session default at every site that previously sent sessionID — openai/setCacheKey, azure, opencode gpt-5, venice, and openrouter (prompt_cache_key). Missing or empty values fall back to sessionID, so behavior is unchanged unless configured. The host config decides the scope; for Replo the harness will set it to a user-scoped value in the generated opencode config (a single user's traffic sits well under the 15 req/min per-key ceiling).

@ai-sdk/openai@3.0.53 forwards promptCacheKey first-class — verified in the installed package (dist/index.d.ts), not assumed.

How did you verify your change?

  • bun test test/provider/transform.test.ts — 294 pass (3 new tests: override for openai, empty-string fallback to sessionID, openrouter prompt_cache_key).
  • tsgo --noEmit clean across the workspace (pre-push turbo typecheck, 30/30).

🤖 Generated with Claude Code