Leave room for the extended thinking budget in Anthropic's default `max_tokens`

Merged
DO
1mo ago
+517-2

No issue: found while live-verifying #8812. Related to #7468 and #2553, which are about the 4096 default being too low in general and are left open.

Why we make these changes

AnthropicModel sends max_tokens=4096 when none is set. Anthropic counts an extended-thinking budget_tokens toward max_tokens and rejects a request whose max_tokens isn't greater than the budget. Unified thinking=True, 'medium', 'high' and 'xhigh' map to budgets of 10000ā€“32768 on models without adaptive thinking (Haiku 4.5, Sonnet 4.5, Opus 4.5 and older), so each of them failed with a 400 unless max_tokens was also set. The same applied to any anthropic_thinking budget of 4096 or more, including the 10000 budget in our interleaved-thinking docs example. Bedrock's Converse path doesn't send a default maxTokens and works; verified live.

New public surface

None.

User-visible behavior

agent = Agent('anthropic:claude-sonnet-4-5')
agent.run_sync('...', model_settings={'thinking': 'high'})
# before: max_tokens=4096, budget_tokens=16384 -> 400 "`max_tokens` must be greater than `thinking.budget_tokens`"
# after:  max_tokens=20480 (16384 + 4096), budget_tokens=16384 -> works

Without max_tokens, an extended-thinking request now sends the budget plus the usual 4096 for the answer. An explicit max_tokens is sent as is, and requests without thinking or with adaptive thinking still send 4096. A default above the SDK's non-streaming limit uses the existing transparent streaming fallback in AnthropicModel.request().

Verification

  • tests/models/anthropic/test_max_tokens.py::test_default_max_tokens_leaves_room_for_the_thinking_budget: VCR, recorded live on Sonnet 4.5 and Sonnet 4.6. Covers no thinking, unified extended thinking, an explicit budget, adaptive thinking, and an explicit max_tokens. It asserts the wire max_tokens and thinking. The two extended-thinking cases fail when the fix is reverted.

What changes for existing users

Extended-thinking requests without max_tokens that used to 400 now work. The ones that already worked, with budgets under 4096, now get 4096 more tokens of room for the answer.

Checklist

  • No breaking changes in accordance with the version policy.
  • Any permitted compatibility impact has a label, warning, migration, release note, and exact API-check waiver.
  • PR title is fit for the release changelog.

šŸ¤– Generated with Claude Code