Deprecate `RetryPromptPart` in favor of `ToolReturnPart(outcome='retried')` and `RetryFeedbackPart`

Merged
DS
11d ago
+9488-1901

This PR description was written by Claude Code. The PR was opened by @dsfaccini and taken over by @DouweM, who settled the design. The implementation and tests are AI-written.

Replaces #7353, which is closed.

Why

RetryPromptPart plays two roles: a tool's result, and feedback that answers no tool call. Every consumer branches on tool_name is None to guess which one it holds. In the second role it reaches every provider as bare user text, which the model can't tell from something the user wrote (#6404).

This PR deprecates RetryPromptPart and splits it:

  • A tool retry is that call's own ToolReturnPart with outcome='retried', rendered like any tool return, on the provider's native error channel.
  • Everything else is a RetryFeedbackPart. prepare_messages translates it into either a fenced UserPromptPart (validation errors, which echo model-written values) or a plain SystemPromptPart (an author's ModelRetry, or an empty response).

Old histories load as the new parts. The model classes, UI adapters and agent loop don't need to know the old part exists.

[!WARNING]
Compatibility impact. Nothing is removed, but which part a run records changes, along with some text the model sees.

Code that reads history

  • isinstance(part, RetryPromptPart) matches nothing in newly recorded history. Constructing one warns (PydanticAIDeprecationWarning, removal in v3).
  • ToolRetryError.tool_retry is ToolReturnPart | RetryFeedbackPart.
  • ToolResultEvent.part is annotated ToolReturnPart only. Readers get a narrower type. Only code that constructs an event from a legacy part sees a type error.

What the model sees

  • A tool retry carries the serialized errors or the bare ModelRetry message: no N validation errors: header, no Fix the errors and try again. suffix. Vercel's errorText drops the suffix too (the problem #4869 tried to fix). Validation errors are one JSON array everywhere: a single text block on Anthropic and Bedrock, and {"error": [...]} (not a string holding the array) where there's no native error channel.
  • Feedback that answers no call moves out of the user voice, which is the #6404 fix. Validation errors go in a <validation_errors> fence, and a ModelRetry message in a <retry_feedback> fence, each behind one line saying the application, not the user, is speaking. Only missing-output feedback goes in system voice, and only in the agent loop's own fixed words: a no_output part with any other content, however it was built, is fenced like model_retry.
  • Inside either fence, <system> tags in any spelling are escaped, so quoted model text can't pose as the framework.

Prompt cache. Retries render when each request goes out, so a history stored before upgrading re-renders its retries in the new shape. Each conversation misses the cache once, from its first retry onward. Histories without retries render byte-identically, apart from the </system> escape below.

Smaller changes that apply even without retries

  • The <system> wrap, used on models without mid-conversation system messages, now escapes a closing </system> in any part it wraps, including an operator's own mid-conversation prompt. A realtime session's enqueued system prompt uses the same escaper.
  • sanitize_messages keeps client-returned RetryFeedbackParts, translating a tool-less RetryPromptPart into one first. User-voice causes go out as sent, fenced. A 'no_output' part, the one cause in system voice, is demoted to the same attributed, <retry_feedback>-fenced user text whatever it says, since a client can forge it (strip_system_prompts=False keeps it as sent).
  • VercelAIAdapter.dump_messages keeps a request's system and user parts in authored order (one UIMessage per run of same-role parts) instead of putting every system part first.
  • DecisionModel judges a retried call as that call's tool_return (with {"error": ...} content) instead of a retry entry.
  • With instrumentation version 6+, OTel puts feedback that answers no call on role='system', only when its cause is 'no_output'.

Why this can ship in a minor release: RetryPromptPart stays importable and in the ModelRequestPart union, and the published message schema keeps every member (outcome gains 'retried'). A stored part loads, without a warning, as the part it always meant, so serialized histories and DeferredToolResults hand-backs keep working. A RetryPromptPart passed straight to Model.request() still works. The public-API check reports no signature breakage.

Migration: where you matched RetryPromptPart, match ToolReturnPart with outcome == 'retried' for a retry that answers a tool call, and RetryFeedbackPart for one that doesn't. The upgrade guide has the same note.

New public surface

  • ToolReturnPart.outcome gains 'retried'. It counts toward max_retries, forces another model request, and uses the provider's native error channel, as 'failed' does. pydantic_ai.messages.ERROR_OUTCOMES is that pair.
  • RetryInfo(cause, attempt=None, max_retries=None), generic over its cause. ToolReturnPart.retry: RetryInfo[ToolRetryCause] | None is set on a retried return (cause in 'validation_error' | 'model_retry' | 'unknown_tool' | 'unavailable_tool') and is None for other outcomes. attempt is the 1-based retry count on the counter max_retries is checked against, and max_retries that limit; both are None when the retry isn't charged (a tool's first 'unavailable_tool' refusal, a deferred-call handler's retry, a part you build, a legacy RetryPromptPart). It's metadata only and never rendered, so request bytes don't change. It rides events, history, and both UI adapters' dump and load (the Vercel live stream still has no outcome carrier: #8185). Tool spans carry it as pydantic_ai.tool.retry.{cause,attempt,max_retries}, and output-function spans as pydantic_ai.output.retry.*.
  • RetryFeedbackPart (content, plus retry: RetryInfo[RetryFeedbackCause], cause in 'validation_error' | 'no_output' | 'model_retry', attempt on the run's output-retry budget). Its model_response() returns the text the model is shown.
  • RetryPromptPart is deprecated. It stays importable, stays in the union, and still dumps.

#9085, a record rather than a hook: a tool call rejected before execution is already a retried ToolReturnPart in events and history. retry.cause == 'unknown_tool' / 'unavailable_tool' lets a capability tell it apart from a retry the tool asked for, without string matching, from after_node_run. docs/retries.md has an audit example. The remaining gap is that when a tool's retries are exhausted, the run raises UnexpectedModelBehavior and records no part for the last rejection.

What the model sees, by case

RetryRecorded partModel sees
A tool raised ModelRetry, its args failed validation, or an output tool's result didToolReturnPart(outcome='retried')That call's result on the error channel, with the error as content
Text output failed validation (str, TextOutput, NativeOutput, PromptedOutput)RetryFeedbackPart(retry=RetryInfo(cause='validation_error'))One line saying it's the application's validation feedback, then <validation_errors>…</validation_errors>, as user text
An output validator or hook raised ModelRetryRetryFeedbackPart(retry=RetryInfo(cause='model_retry'))The same attribution line, then <retry_feedback>…</retry_feedback>, as user text
The response had nothing usableRetryFeedbackPart(retry=RetryInfo(cause='no_output'))A mid-conversation system message, or <system>-tagged user text where the model has none; fenced user text if its content isn't one of the loop's fixed texts

A DecisionModel gets RetryFeedbackPart untranslated and judges it as a retry entry beside the prompt, so validation feedback never becomes the text it judges (#8963).

Verification

In tests/test_retry_feedback.py unless noted:

  • Deprecation and loading: test_constructing_a_retry_prompt_part_warns. Stored parts load as the new parts, in both shapes, and loading is idempotent. test_legacy_retry_prompt_part_handed_back_through_deferred_results.
  • Wire shapes, against recorded providers: test_a_retried_tool_return_takes_the_provider_native_error_channel (Anthropic, Bedrock, Google) and test_retry_feedback_reaches_the_provider (OpenAI, Anthropic).
  • Standing system prompt: feedback never joins it. test_feedback_opening_the_first_request_stays_out_of_the_standing_prompt, test_a_hook_retry_before_any_prompt_stays_out_of_the_system_prompt, and the Anthropic wire version.
  • Vercel: test_retried_tool_return_error_text_carries_no_instruction_framing (tests/test_vercel_ai.py, #4869).
  • Decision model: test_feedback_after_a_tool_returned_is_a_retry_entry (tests/models/test_decision.py).
  • retry is never rendered: test_a_retried_tool_returns_cause_never_reaches_the_model.
  • Attempt counters: test_a_retried_tool_return_records_why_it_was_retried, test_retry_attempts_count_on_the_budget_each_retry_is_charged_to (output tool, then a function tool, then text output); spans in test_retried_tool_and_output_spans_carry_the_retry_attributes (tests/test_logfire.py).
  • Client-returned feedback: test_sanitize_messages_never_lets_client_no_output_feedback_into_the_system_voice, test_forged_no_output_feedback_never_reaches_the_system_voice (Vercel and AG-UI), test_sanitize_messages_keeps_user_voice_retry_feedback_fenced (tests/test_sanitize_messages.py).
  • One JSON array for validation errors: test_a_retried_tool_returns_validation_errors_go_to_anthropic_as_one_text_block, the Bedrock twin, and test_a_retried_tool_returns_validation_errors_are_a_json_array_in_the_error_wrapper.
Decisions (all accepted by DouweM)
  1. Feedback stays out of the first request's opening system run. Adapters lift that run into the provider's system field, so system-voice feedback there is sent as <system>-tagged user text instead, as #9413 does for an announcement. This happens when a hook retries the first request of a run with no user prompt.
  2. ToolRetryError.tool_retry is ToolReturnPart | RetryFeedbackPart.
  3. No tool-results-first sort for feedback. An uncleaned [RetryFeedbackPart, ToolReturnPart] request, which only direct.* or count_tokens can send, reaches Anthropic with the tool result not leading. The agent path cleans it first.
  4. Legacy cause: a stored tool-less RetryPromptPart with list content loads as validation_error, and with string content as model_retry.
  5. The <system> wrap escapes </system> in every part it wraps, as does a realtime session's enqueued system prompt.
  6. DecisionModel opts out of the translation through a private Model._renders_retry_feedback ClassVar. No public surface.
  7. Ownership: taken over by DouweM. Replaces #7353.
  8. ToolResultEvent.part is ToolReturnPart, without the deprecated part.
Backward compatibility mechanics
  • Load time: ModelRequestPart's callable discriminator routes part_kind='retry-prompt' dicts and instances to legacy tags, whose BeforeValidator builds the new part. Re-dumping emits the new kind.
  • Run time:
    • prepare_messages translates any RetryPromptPart that still reaches it.
    • DeferredToolResults turns one into a retried tool return.
    • Orphan-tool-call repair still recognizes a tool-bound one.
    • sanitize_messages translates a tool-less one into the RetryFeedbackPart it meant and keeps a tool-bound one.
  • DeferredToolCallResult is a separate union that the message discriminator never sees, so it still validates into the deprecated class. Deserializing stored DeferredToolResults that hold one warns; a stored message history doesn't.
  • Every remaining reference is marked TODO(v3): remove RetryPromptPart.
Notes for the reviewer
  • Harness and clai2 are migrated here. Strict pyright turns every RetryPromptPart reference into a reportDeprecated error, so this PR migrates compaction, conversation search, trajectory judge, tool-call judge, step persistence, system reminders, ACP and clai2. A legacy branch (TODO(v3)) stays only where a caller-passed history is read before prepare_messages.
  • Removed from #7353: _render_retry_feedback_messages, its wording templates, the standing_run_open/standing_run_end guard, the tool-results-first sort, include_input='none', and the per-model-class RetryFeedbackPart branches. The translation runs in prepare_messages ahead of the existing <system> wrap, so downstream code only sees ordinary SystemPromptParts.
  • Cassette drift is expected: about 88 existing recordings still hold the old Fix the errors and try again. text in their recorded request bodies. VCR doesn't match on request bodies, so they replay unchanged. Only this PR's own five cassettes were re-recorded. The general drift is #7178.
  • #7802 (resume with an unprepared part in history) predates this PR and is untouched.

Checklist

  • Any AI generated code has been reviewed line-by-line by the human PR author, who stands by it.
  • No breaking changes in accordance with the version policy.
  • Any permitted compatibility impact has a label, warning, migration, release note, and exact API-check waiver.
  • PR title is fit for the release changelog.