Default `BedrockConverseModel`'s `maxTokens` to Claude's maximum output, streaming such requests behind the scenes

Merged
DO
11d ago
+7803-73216

Why we make these changes

Without max_tokens, BedrockConverseModel left inferenceConfig.maxTokens out, and Bedrock caps Claude Opus 5.5 at 4,096 output tokens. Opus 5.5 always thinks, and thinking counts toward that cap, so a run could end with finish_reason='length' and no text. It now sends the profile's anthropic_max_output_tokens, as AnthropicModel does since #9298.

request() streams those requests behind the scenes. A non-streaming Converse request that outlasts the client's read timeout (300s by default) is retried by botocore 5 times, regenerating the response each time, and then fails. Streamed, the timeout limits the wait between chunks. The streamed response is built by BedrockStreamedResponse, which tests/models/bedrock/test_stream_parity.py checks against Converse.

[!WARNING]
Compatibility impact: run() requests to Claude Sonnet 4.5 and later without max_tokens now send 64,000 or 128,000 maxTokens and use ConverseStream. That needs the bedrock:InvokeModelWithResponseStream IAM permission, which streamed runs already need, and reserves more tokens-per-minute quota up front. A server error that arrives inside the stream isn't retried by botocore, as on streamed runs; it raises ModelHTTPError for Pydantic AI's retries or FallbackModel to handle. Tool call args in those results are the streamed JSON string rather than a dict, as on streamed runs; args_as_dict() gives the same value.

Why this can ship in a minor release: it removes an artificial output cap that made Opus 5.5 runs fail, matching #9298 on AnthropicModel.

Migration: Set max_tokens to keep Bedrock's default limit and non-streaming Converse requests.

New public surface

None.

User-visible behavior

agent = Agent(BedrockConverseModel('us.anthropic.claude-opus-5-5'))
result = agent.run_sync('Write a detailed 5,000-word history of the printing press.')
# before: finish_reason='length', output_tokens=4096 (or UnexpectedModelBehavior with no text)
# after:  finish_reason='stop', output_tokens=16469, one ConverseStream request

Verification

  • test_bedrock_default_max_tokens (recorded on Opus 5.5 and Nova) pins the wire maxTokens and operation for the default, an explicit max_tokens, and a non-Claude model. test_bedrock_default_max_tokens_stream_closed_on_failure checks the stream is closed on a mid-stream error and on cancellation.
  • Live: Bedrock accepts each model's maximum on every Claude model that gets it (Fable 5 is blocked by the test account's data retention mode); 128,001 is rejected on Opus 5.5.
  • Re-recording the requests that now stream exposed two pre-existing Bedrock changes, reproduced on the unchanged Converse path on Sonnet 4.5 and Opus 5.5: Claude rejects S3 documents in tool results, and a conversation with documents no longer reads the previous turn's cache. The multimodal matrix and the document cache test now pin that.

What changes for existing users

Claude Sonnet 4.5 and later on Bedrock can produce responses up to the model's maximum without max_tokens, and run() streams those requests.

Checklist

  • No breaking changes in accordance with the version policy.
  • Any permitted compatibility impact has a label, warning, migration, release note, and exact API-check waiver.
  • PR title is fit for the release changelog.

šŸ¤– Generated with Claude Code