Don't force output tools on Anthropic requests that think, so Claude keeps thinking with a structured `output_type`

Merged
DO
1mo ago
+1783-648

Stacked on #8808, which is this PR's base branch; it will retarget to main once that merges.

Why we make these changes

Anthropic accepts a forced tool_choice alongside adaptive thinking, but answers it without thinking. Tool Output forces the output tool on every request that can't end with text, so a bare structured output_type silently switched thinking off on every step of a run. Probed live on Opus 5, Sonnet 4.6, Sonnet 5 and Fable 5: forced (any or tool) returned 0 thinking blocks in 15 of 15 responses; auto kept thinking. #8772 measured the effect: more turns, more searches, more tokens.

Which mode a model got also depended on whether a thinking setting was passed, not on whether the request thinks (#8640). Claude Opus 5 and later, Sonnet 5, and the Fable and Mythos models think with no setting at all. On Opus 5 and Sonnet 5, thinking=False didn't turn it off either (#8199).

The rule is now: don't force a tool call Pydantic AI chose itself when the request will think.

New public surface

  • ModelProfile.forced_tool_choice_disables_thinking. Anthropic sets it on every Claude model, so it applies on every route that serves one.

User-visible behavior

agent = Agent('anthropic:claude-opus-5', output_type=City)  # no thinking setting
# before: Tool Output, tool_choice={'type': 'any'} on every request -> no thinking blocks
# after:  Native Output (output_config.format), tool_choice omitted -> Claude thinks as usual
  • A bare structured output_type uses Native Output whenever the request thinks: with a thinking setting, or with none on a model that thinks by default. It falls back to Prompted Output under extended thinking on models without JSON schema output, and on Bedrock keeps Tool Output unforced where Bedrock has no native structured output (for example, Opus 5 and Sonnet 5).
  • A forced choice Pydantic AI resolved itself falls back to auto while the request thinks. An explicit tool_choice='required' or tool list is still sent where the API accepts it.
  • An explicit ToolOutput(...) under extended thinking, or on a model that can't force, no longer raises. The output tool is offered under auto, and a text answer is retried.
  • thinking=False sends {'type': 'disabled'} on Opus 5 and Sonnet 5 (direct and Bedrock); models that can't turn thinking off ignore it.
  • claude-mythos-preview gets adaptive thinking.
  • Routes that serve Claude through an OpenAI-compatible API (OpenRouter, Vercel, GitHub Copilot, Heroku, LiteLLM, Snowflake) follow the same rule, because the flags live on the Claude profile: there, Native Output means a JSON-schema response_format. It has been verified on the direct API and Bedrock only. On OpenRouter, an explicit forcing tool_choice still raises when thinking was asked for explicitly, since OpenRouter would drop the reasoning, but not for thinking the model does by default.

Verification

  • tests/models/anthropic/test_output.py::test_structured_output_mode_follows_whether_the_request_thinks: VCR, recorded live across Opus 5, Sonnet 5 (thinking=False), Sonnet 4.6 with and without thinking, Opus 5.5 and Sonnet 5.5. Pins the wire request and whether a thinking block came back.
  • tests/models/test_anthropic.py::test_anthropic_adaptive_thinking_keeps_tool_output_unforced, ::test_anthropic_output_tool_with_thinking (re-recorded)
  • tests/models/test_bedrock.py::test_bedrock_adaptive_thinking_keeps_tool_output_unforced, ::test_bedrock_opus_5_thinking_offers_output_tool_unforced, ::test_bedrock_output_tool_with_extended_thinking_is_unforced (re-recorded live)
  • tests/test_thinking.py, tests/profiles/test_anthropic.py::test_model_profile_thinking_defaults, tests/models/test_openrouter.py::test_openrouter_structured_output_mode_follows_whether_forcing_stops_thinking

What changes for existing users

Claude users with a bare structured output_type whose requests think now get Native Output instead of forced Tool Output. That applies to Opus 5 and later, Sonnet 5, Fable and Mythos without a thinking setting, and to any Claude model with one. To keep forced Tool Output, pass thinking=False where the model allows it.

Checklist

  • Any AI generated code has been reviewed line-by-line by the human PR author, who stands by it.
  • No breaking changes in accordance with the version policy.
  • Any permitted compatibility impact has a label, warning, migration, release note, and exact API-check waiver.
  • PR title is fit for the release changelog.

🤖 Generated with Claude Code