Raise `AnthropicModel`'s default `max_tokens` from 4096 to 16384 on Claude Sonnet 4.5 and later
Why we make these changes
Anthropic requires max_tokens, and AnthropicModel sent 4096 when it wasn't set. Thinking counts toward max_tokens, and Claude Opus 5 and later, Sonnet 5 and Fable think by default. A run could spend the whole budget thinking and fail with UnexpectedModelBehavior, or have its answer cut short (#7468). #1979 once used each model's maximum output. #1994 reverted it because the Anthropic SDK refuses non-streaming requests with max_tokens above about 21,333, which #4496 has since handled with a transparent streaming fallback.
The default is now 16384 on Claude Sonnet 4.5 and later. That matches Anthropic's guidance for non-streaming requests, keeps run() a single non-streaming request, and fits every model that gets it. Models older than Claude Sonnet 4.5 keep 4096. They reject a request whose input plus max_tokens exceeds the context window, and Claude 3 and 3.5 also have a maximum output below 16384. An extended thinking budget still raises the default to leave 4096 beyond it (#8986).
Model-maximum defaults, as LangChain, LiteLLM and the Vercel AI SDK use, would need a per-model table. They would also stream every run() and reserve up to 128K tokens of Bedrock quota per request.
[!WARNING]
Compatibility impact: Through AnthropicModel on Bedrock (AsyncAnthropicBedrock), requests to Claude 4.5 and later that don't set max_tokens now reserve 16384 output tokens of Bedrock's TPM quota up front instead of 4096. Bedrock refunds the unused part afterwards. Deployments running close to their quota may be throttled sooner.
Why this can ship in a minor release: 4096 was a placeholder for a field Anthropic requires. It was too low for models that think by default (#7468). The direct Anthropic API doesn't count max_tokens toward rate limits, and BedrockConverseModel sends no default.
Migration: Set model_settings={'max_tokens': 4096} to keep the previous limit.
New public surface
AnthropicModelProfile.anthropic_rejects_max_tokens_beyond_context_window: set for Claude 3, 3.5, Sonnet 4 and Opus 4/4.1 (direct, Bedrock and Vertex ids), which keep the 4096 default.
User-visible behavior
Request without max_tokens | Before | After |
|---|---|---|
| Claude Sonnet 4.5 and later, no thinking or adaptive thinking (default on Claude 5) | 4096 | 16384 |
| Claude Sonnet 4.5 and later, extended thinking with budget ā¤ 12288 | budget + 4096 | 16384 |
| Extended thinking with a larger budget, or any model older than Sonnet 4.5 | unchanged | unchanged |
An explicit max_tokens is sent as is.
Verification
tests/models/anthropic/test_max_tokens.py::test_default_max_tokens: VCR, recorded live on Sonnet 4.5, Sonnet 4.6 and Opus 5. It asserts the wiremax_tokens, and pins that the'xhigh'case streams.::test_default_max_tokens_on_models_that_reject_overflowing_the_context_window: mocked, because these models can no longer be recorded. The API has retired them, and Bedrock only serves them to accounts that used them recently. Covers direct, Bedrock and Vertex ids.- Live, unrecorded:
- A structured
output_typeon all 13 Claude models on the direct API, with no thinking setting,thinking=False,thinking='high'andToolOutputwith thinking. All 52 runs succeeded. - Near the context window: Haiku 4.5 accepts 192K input tokens plus 16384, direct and on Bedrock. Bedrock Sonnet 4 rejects 192K plus 16384 (
Input is too long for requested model.) and accepts 192K plus 4096.
- A structured
What changes for existing users
Claude Sonnet 4.5 and later can produce longer responses when max_tokens isn't set. Older models behave as before. For the Bedrock quota effect, see the warning above.
Checklist
- No breaking changes in accordance with the version policy.
- Any permitted compatibility impact has a label, warning, migration, release note, and exact API-check waiver. (No public API is removed or changed, so no API-check waiver applies.)
- PR title is fit for the release changelog.
š¤ Generated with Claude Code