#9405 Record every mid-run tool population change as a `ToolAvailabilityDeltaPart`

• Closes #7251

Before this PR, only three things recorded a ToolAvailabilityDeltaPart: tool search, load_capability, and ToolReturn.tools. Every other mid-run change to the tool set rewrote tools, the first prompt-cache section, and gave the model no explanation for it. This PR adds one diff point that catches all of them. Each step's immediately available function tools are compared against what the run started with plus what history's deltas already name. Each newcomer gets a delta on that step's request and goes on the provider's tool-addition channel, so tools doesn't change.

[!WARNING]
Compatibility impact: Code that inspects message history or outgoing requests may see a difference after a tool joins mid-run:

• Mid-run tool additions now record a ToolAvailabilityDeltaPart in history, so it shows up in all_messages(), new_messages(), UI adapter dumps and event streams (AG-UI streams it as a pydantic_ai_tool_availability_delta activity message).
• On OpenAI Responses and Anthropic, a newly appeared tool no longer joins tools[]. It arrives as an additional_tools input item (Responses) or as a defer_loading declaration plus a tool_addition block or synthesized tool_reference exchange (Anthropic). Code that inspects the request's tool list sees it in a different place. ModelRequestParameters gains an introduced_tool_names field marking these tools.
• On some Anthropic models (Claude Opus 4.8, for example), a run's first deferred declaration adds a one-time preamble of about 93 tokens. A run whose first deferred tool is a mid-run newcomer moves its cache prefix once on that request. main moves it there too, by appending the tool to tools[].
• AG-UI: @ag-ui/client doesn't send activity messages back, so when the frontend holds the history the delta is lost on the round trip. The next run sends the tool in tools[], so the prefix moves on that run (see Records carried as activity messages).
• Temporal: a change made with add_function() from inside a tool runs in an activity, which replay doesn't re-execute, so replay won't reproduce the change or its delta.

Why this can ship in a minor release: No public API is removed or changed. The tools stay callable exactly as before (ToolDefinition.defer_loading and RunContext.is_tool_available are untouched); only where a mid-run tool travels on the wire changes, and a new history part appears. ToolAvailabilityDeltaPart is an existing part type that tool search and capability loading already record, and the version policy permits new message parts in history and new optional fields such as introduced_tool_names in minor releases. How a request lays out its tools on the wire was never a documented contract.

Migration: Code that filters or renders history should handle ToolAvailabilityDeltaPart the way it already does for tool-search reveals. Code that reads the tool list off outgoing requests should also read introduced_tool_names or the provider's addition channel. Under durable execution, drive mid-run tool changes from workflow-side code, such as a prepare_tools function or a dynamic toolset that reads run state, rather than add_function() inside a tool. AG-UI apps that need the record across runs should keep history on the server or send the activity messages back.

Reproduction (before this PR)

These FunctionModel probes record the function tools sent on each request:

Cause	Request 1 tools	Request 2 tools	Deltas in history
FunctionToolset.add_function() inside a tool	unlock	unlock, later	none
dynamic toolset (@agent.toolset)	unlock	unlock, later	none
MCP add_tool + tools/list_changed	unlock	unlock, later	none
PrepareTools admits later from step 2	unlock	unlock, later	none
approval-gated filtered() unlock	escalate	escalate, admin_action	none
per-tool prepare=	unlock	unlock, later	none
control: ToolReturn.tools on a defer_loading tool	unlock	unlock, later	[['later']]

With this branch, every row except the approval-gated one records [['later']]. The approval-gated unlock always lands on the first request of the next run, because approval ends a run with DeferredToolRequests. It's left to the cross-run baseline follow-up.

Design

• Diff point. _record_tool_population_changes runs in ModelRequestNode._prepare_request on the parameters before_model_request hooks return, so it diffs the tools the request actually carries. The delta is appended to the step's request, or to the persisted tail when a hook ends the request in a request-only message (_history_recording_targets, shared with on_change='append' instruction deltas). It's written back to history on the same rail as ToolReturn.tools, not via enqueue. Continuations (_prepare_resume_request) mark newcomers but never append.
• What the model already knows. This is the new run-scoped GraphAgentDeps.established_tool_names plus discovered_tool_names. The established set is fixed at the run's first request: its visible function tools, minus any tool history already shows arriving by delta. Already-recorded deltas are read from the history the before_model_request hooks return, in its post-compaction window. So retries and replays over persisted history never re-announce, and after compaction, or a hook that removed the delta, a tool is announced once more without changing tools.
• No baseline to protect. When the run started with no plainly visible tools, or every established tool has left the population, the tools present establish tools[] with no delta. This also keeps an Anthropic request from carrying only deferred tools, which it rejects. resolve_request_tools repeats the check on what survives provider resolution, so an unsupported optional native tool doesn't count.
• Streaming. Each recorded delta is also emitted as a ToolAvailabilityDeltaEvent, so the Vercel AI and AG-UI streams carry it and a client that rebuilds history from the stream keeps it.
• Projection. ModelRequestParameters.introduced_tool_names marks newcomers. resolve_request_tools resolves a newcomer like a revealed deferred tool, but only once its delta is in the outgoing history: via_history on OpenAI Responses, deferred on Anthropic, visible plus the prose announcement elsewhere. If the delta is missing from the outgoing history, the tool stays visible: it's callable either way.
• Callability doesn't change (#8071). ToolDefinition.defer_loading stays as authored, and is_tool_available never reads the new field.
• Deferred newcomers are not announced. Announcing isn't loading, and the local search corpus already rescans every step.
• Out of scope: removals (#6985), same-name schema changes (#7243), and realtime (#7288, #7191). Realtime resolves newcomers to visible, which is the right session.update shape once #7191 lands.

Tests (tests/test_tool_population_diff.py)

For each of the five causes that happen within a run, the tests check that:

• exactly one delta is recorded, next to the tool result that caused it;
• on OpenAI Responses, tools is byte-identical across all three requests and the message prefix is preserved;
• on Anthropic, the non-deferred tool entries are byte-identical and the newcomer is only a tail deferred declaration plus its reveal, for both claude-opus-4-8 (tool_addition) and claude-sonnet-4-6 (tool_reference).

Live recordings confirm the providers accept the projection: claude-opus-5-5 (tool_addition), claude-sonnet-5 (synthesized tool_reference) and gpt-5.6 Responses (additional_tools).

Other tests cover replay over persisted history, a retry, request-only hook tails, re-announcing after compaction, re-recording a delta a hook removed, a hook-filtered tool, a continuation, rotating out every visible tool (with and without an unsupported optional native tool), Vercel AI and AG-UI round trips through a client-rebuilt stream, the per-channel visibility matrix, the conservative direction (nothing changes, so no delta, no introduced_tool_names, every tool visible), and the no-channel announcement.

Decisions (settled with @DouweM)

• introduced_tool_names is a separate field rather than flipping defer_loading.
• The check is built into the graph, not exposed as a capability hook.
• Tool announcements stay plain prose, not #8188's <context id> tags; #10547 leaves out a name that carries a framing tag. #8188's measurements were about following replaced state, which announcements don't have.
• Deferred newcomers rely on the search corpus picking them up, not on an announcement.
• Follow-ups:
  • a cross-run tool baseline recorded in history, which also catches the approval-gated cause;
  • ToolReturn.tools accepting definitions and callables.

Prompt cache, measured live on Anthropic

The probe sends about 11k tokens of cached instructions with tool calls unlock → later. Request 2 is the first after later appears:

Model	Scenario	Request 2 cache write / read
claude-opus-4-8	nothing changes	0 / 11172
claude-opus-4-8	newcomer, a deferred tool already declared	0 / 11464
claude-opus-4-8	newcomer, no deferred tool before	11202 / 0
claude-sonnet-4-6	newcomer, no deferred tool before	0 / 6846

On Opus 4.8, the first deferred declaration in a run adds the one-time preamble described in the warning above, whether the reveal is a tool_addition block or a tool_reference result. The docs say so.

Also in this PR

• tests/durable_exec/temporal/test_durability.py::test_population_change_delta_is_recorded_once_and_replays: the delta is recorded once in a workflow, and replaying that workflow's history raises no non-determinism error.
• Docs: a "Tools that appear mid-run" section in tools-advanced.md, a row in the on-demand cache table, a paragraph in toolsets.md, and notes in the caching guide's "What invalidates a cache" and the AG-UI activity-records section.
• The agent skill's TOOLS-ADVANCED.md reference and its index.
• test_adding_tools_during_run now records the delta.

Depended on by

• #10546's toolsets rule depends on this.
• CodeMode (#10906, the #8965 fix) needs nothing from it: it keeps sandboxed tools off the wire and appends to its own catalog.

Checklist

• No breaking changes in accordance with the version policy.
• Any permitted compatibility impact has a label, warning, migration, release note, and exact API-check waiver.
• PR title is fit for the release changelog.

🤖 Generated with Claude Code