Carry non-native `ThinkingPart`s in a user turn for models that imitate assistant formatting
- Closes #5869
The bug. After a few turns, Claude starts writing <thinking>…</thinking> into the answers users read. Once those answers are saved to history, every later turn reinforces it. This happens whenever a ThinkingPart can't use Anthropic's native reasoning channel: reasoning from another model in a FallbackModel chain, or Claude's own reasoning whose signature was lost in a history processor or a storage/UI round trip. We then replay it as <thinking>-tagged text in the assistant turn, and Claude copies the format.
The cause is the turn, not the tag. Six prior turns with reasoning replayed, then a fresh question; "leaks" = a thinking tag in the visible answer, n=4 per cell (2026-10-15):
| model | reasoning in the assistant turn | same text in its own user turn |
|---|---|---|
claude-sonnet-5-5, claude-haiku-4-5, claude-opus-4-5 | 4/4 | 0/4 |
claude-opus-5-5, claude-fable-5-1 | 0/4 | 0/4 |
Earlier runs agree: claude-opus-4-5 ~12/12 → 0/16, and claude-opus-4-8 4/4 → 0/4. Changing the tag doesn't help (Opus copies annotations and attributes verbatim); only moving the text out of the assistant turn does.
What changes
- A new
ModelProfile.mimics_assistant_message_formattingflag, on for the whole Anthropic family. Gating by model name would silently regress if a future model brings the behavior back. - When it's set,
AnthropicModelandBedrockConverseModelsend such reasoning as its ownusermessage directly ahead of the assistant turn it belongs to. Signed same-provider thinking still goes through the native channel, and other models are untouched. - The carried text is wrapped as
<assistant_thinking model="gpt-5.6" provider="openai">…</assistant_thinking>:- the attributes come from the enclosing
ModelResponse, and each is left off when unknown; - the tag is rendered per request and never stored in history;
- attribute values are HTML-escaped, and any spelling of a closing tag inside the reasoning is neutralized.
- the attributes come from the enclosing
- A response that called a server tool it never resolved needs no special case here:
mainalready drops the unpaired call before it reaches the wire (#7504), and the placement tests pin that shape.
Trade-offs (accepted)
- Asked who wrote the reasoning, Claude claims it as its own, with or without the attribution. The tag is provenance for the record, not a change in model behavior.
- Model-produced text rides a user-role message. It is attributed and escaped so it can't pose as the user's own words or break out of the wrapper.
- Dropping non-native reasoning was the alternative; we keep it because the next turn usually needs that context. Opt out per model with
mimics_assistant_message_formatting=False.
Cache. The carried turn is placed during mapping, ahead of any mid-conversation system entry, so that entry keeps its cache boundary. On Bedrock it goes in front of an explicit trailing CachePoint, so it never costs a cache point. Checked live with Caching(): three consecutive requests read 11099 → 11135 cached tokens on claude-sonnet-5-5 and 9582 → 9600 on claude-haiku-4-5, writing only the new tail each time.
Follow-ups. Claude served through OpenAI-compatible routes (vercel, litellm, heroku, snowflake) goes through OpenAIChatModel, which doesn't read the flag: #7419. Related: #9017 labels whole assistant turns from another model; once it adds a normalized model identity, the model/provider attributes here should render from it.
Tests
tests/test_thinking_placement.py: the mapping across every history shape and both adapters, plus the opt-out and the escaping.tests/models/anthropic/test_carried_thinking.py: Anthropic accepts the hardest shapes live (after a tool result, with extended thinking, and overAsyncAnthropicBedrock); the body is read off the wire.tests/test_code_execution_files_vcr.py:container_uploadblocks on the carried turn are accepted, and the file reaches the container.tests/models/anthropic/test_thinking_tag_mimicry.py: recorded model behavior. Assistant-turn tags leak and the user-turn placement doesn't, now includingclaude-sonnet-5-5.
Checklist
- Any AI generated code has been reviewed line-by-line by the human PR author, who stands by it.
- No breaking changes in accordance with the version policy.
- PR title is fit for the release changelog.
🤖 Generated with Claude Code