#42 feat(opencode): fall back to a configured sibling model when a provider is down

Do not merge. Opened so we can look at the actual size of doing model fallback inside opencode instead of around it.

What this does

When a step's LLM call fails with a status the coordinator lists (or never reaches a provider at all), and the same-provider retries are spent, the step swaps to the mapped sibling model and continues. The failing provider is marked degraded for the configured cooldown, so later steps in the same sandbox start on the healthy route instead of paying the retries again. The assistant message is updated to name the model that actually answered.

Config comes from REPLO_OPENCODE_FALLBACK_CONFIG, which the coordinator already injects into every sandbox (apps/cloudflare-agent-coordinator/src/lib/opencode-fallback-config.ts in andytown). Nothing in this fork read it before. No env var means no behaviour change.

Also fixes a caching bug seen in prod: applyCaching marked an empty trailing text part with cache_control, which Anthropic rejects (cache_control cannot be set for empty text blocks). An empty tail now falls through to the message-level marker.

Shape of the change

File	Lines	What
src/session/fallback.ts (new)	+120	Parse config once, qualifies(error), next(), degraded-provider map with cooldown, healthy() chain walk with cycle guard
src/session/retry.ts	+28 / -11	policy() gains a provider getter, an optional same-provider attempt cap, and a fallback hook that runs once retries are exhausted or the error is non-retryable
src/session/processor.ts	+41 / -5	Resolves the fallback model via Provider.Service, streams with ctx.model instead of the input model, rewrites providerID/modelID on the assistant message, logs [model-fallback]
src/session/prompt.ts	+7 / -1	Each step resolves SessionFallback.healthy() before getModel
src/provider/transform.ts	+4	Empty-text guard
tests	+315 / -3	test/session/fallback.test.ts (config, qualification, cooldown, cycle), a retry-policy handoff test, and an end-to-end processor test: two 503s from the test LLM server, third call answered by the fallback provider, message stamped fallback/fallback-model, test marked degraded

What happens when both routes are down

The coordinator map is bidirectional for the Claude transport pair (openrouter/anthropic/claude-sonnet-5 <-> anthropic/claude-sonnet-5). That cannot loop:

• Inside one step the number of swaps is capped by maxFallbackAttempts (2 in the coordinator config). With both routes down the sequence is: OpenRouter fails, 3 same-route retries at 2 s / 4 s / 8 s, swap to direct Anthropic, 3 retries, swap back to OpenRouter (second and last swap), 3 retries, then the error surfaces as an ordinary failed turn. Roughly 45 s of waiting, then it stops. Set the cap to 1 to skip the bounce back.
• Across steps the healthy-route walk keeps a visited set. When every route in a cycle is cooling down it returns the model that was asked for, so the next step starts where it would have without this change and fails the same way.
• Same-model retries are unchanged when no fallback config is present, so upstream opencode behaviour is untouched.

Both cases are covered by test/session/fallback.test.ts (cycle) and the retry-policy test (swap cap, fresh budget after a swap).

Decisions worth a look

• Fallback lives in the retry schedule, not a plugin. The schedule already owns "what happens after a failed attempt", so the hook is one extra branch there. The old model-fallback plugin approach needed the plugin to re-run the whole step.
• Per-step swap, per-sandbox cooldown. The swap decision is local to a step (bounded by maxFallbackAttempts); the cooldown is module state so the next step skips the dead provider. It is process memory, so a sandbox restart forgets it, which is fine.
• Which errors qualify: only statuses listed in fallbackOnErrors, whether they arrive as an HTTP status or as OpenRouter's in-stream {code} chunk, plus retryable errors with no status (fetch failed, connection reset). Invalid prompts, quota errors, context overflow and aborts never swap. 402 is not special-cased; the coordinator lists it (replohq/andytown#27790).
• Same-provider retries keep today's behaviour when maxUpstreamRetryAttempts is unset, so upstream opencode semantics are unchanged without our env var.

Partial output from the failed route

Reasoning and text parts a failed attempt already wrote are removed from the assistant message when the step switches models, so the fallback never replays another model's signed reasoning as its own. Tool parts stay, since they record effects that really happened. Covered by a stubbed-provider test that streams partial output and then fails with OpenRouter's in-stream {code: 503}.

Verification

• bun run typecheck clean.
• bun test test/session/fallback.test.ts test/session/retry.test.ts test/session/processor-effect.test.ts: 63 pass.
• bun test test/provider/transform.test.ts: 296 pass, including a wire-level check that the OpenRouter provider never serializes cache_control on an empty text block.
• bun test test/provider: one pre-existing failure (chunkTimeout raises a response stream error when SSE body stalls) that fails identically on dev.

Live run on a sandbox (2026-09-16)

Ran on one of our own internal agent sandboxes with this branch's linux-x64 build swapped in for the fleet binary. OpenRouter traffic was routed through a local stub returning 503, and the fallback map pointed the primary route at direct Anthropic. Prompts were sent straight to opencode's HTTP API on the box.

Check	Result
Startup: s6-svc restart to /global/health 200	1.2 s with this build, 1.3 to 1.5 s with the fleet binary on the same box. No startup tax; the removed harness plugin cost about 52 s at boot, this is compiled into the binary and does one JSON parse of the env var on first use
Memory	no new dependency, no new process, no plugin loader; the module is about 120 lines
Turn 1, primary route down	retries at 2 s, 4 s, 8 s on the primary, then [model-fallback] switched model, real completion from direct Anthropic, 19 s total (nearly all of it the three retries the coordinator config asks for)
Turn 2, same session	no retries, started directly on the fallback route, 3 s total
Stored assistant message	providerID/modelID name the model that answered, on both turns
Prompt cache	first fallback turn wrote the prefix to the new route's cache (one extra cache write), later steps read it

The box was restored afterwards: fleet binary, coordinator env, stub removed.

Review round 1 (Codex, 2026-09-09)

All four findings applied in 19e6164:

1. Fallback model had no retry budget. The schedule now counts attempts since the last swap, so the fallback route gets its own retries and backoff. Covered by the retry-policy test.
2. Cooldown was per provider. Keyed by provider/model route now, so the coordinator's Claude to GPT to Gemini chain on OpenRouter walks correctly. Covered by a test using those exact routes.
3. Empty-text cache fix did not reach the wire. The skip is gone; Claude behind OpenRouter now goes through the same empty-content filter as direct Anthropic. Verified by serializing through the real provider with a stub fetch.
4. Test isolation. Fallback tests reset module state before and after each test.

Follow-ups in andytown (not here)

• replohq/andytown#27790: OpenRouter ↔ direct Anthropic pairs in both directions, 402 in fallbackOnErrors, the eval opt-out, and the pin bump to this release.
• Flip the coordinator default so agent Anthropic traffic routes through OpenRouter.

🤖 Generated with Claude Code

---

Summary by cubic

Adds a configured sibling-model fallback inside session retries when a provider fails, and fixes two prompt-caching bugs that Anthropic and OpenRouter reject. Reads REPLO_OPENCODE_FALLBACK_CONFIG; without it, behavior is unchanged.

Fallback

• After same-provider retries, listed HTTP or in-stream provider errors and retryable transport failures move the step to a mapped route with a fresh retry budget.
• Failed routes cool down per provider/model, so later steps start on a healthy route.
• Fallback rebuilds prompt history for the new model, including compaction; the assistant message names the model that answered, and a failed fallback leaves the requested model on the row.
• Partial reasoning, text, and pending tool parts from the failed route are removed on a swap, cleared tool calls are settled so cleanup doesn't wait out their timeout, and output from earlier steps stays.
• Context overflow, aborts, non-retryable statusless errors, invalid targets, swap limits, a tool that already ran, and failed part reads prevent switching, but the failed route is still marked degraded.
• Config is re-read when the env var changes and cooldowns are cleared; a swap budget of zero disables the feature, including the same-provider retry cap.

Caching

• Empty text blocks are no longer sent with cache markers for Claude through Anthropic or OpenRouter.
• Signed OpenRouter reasoning details survive message filtering.