Add structured citations to `TextPart`
Merged
AD
9d ago+6355+6355+0123456789012345678901234567890123456789-189-189-012345678901234567890123456789
- Part of #3126
Adds TextPart.citations, so an app can show which sources back which part of a model's answer. Anthropic, Gemini, Vertex AI and OpenAI (Chat and Responses, and so OpenRouter and other providers on the OpenAI Chat mapping) return citations in one shape, streamed or not. The response text is never changed. In this PR, requests only change when citations are asked for with the Citations() capability (or ModelSettings(citations=True), which it sets); #7752 also sends some citations back in history.
Citation(
sources=[WebCitationSource(url='https://example.com', title='Example')],
anchor=ContentCitationAnchor(start=0, end=34), # part.content[0:34] is the supported claim
)
How it works, at a glance
provider response
└─ model adapter maps each citation it can check
└─ TextPart(content, ◄── text unchanged
citations=[Citation(sources, anchor)], ◄── same shape for every provider
provider_details={'citations': [raw entries]}) ◄── everything else the provider returned
| Provider | What's cited | How to turn it on | Anchor |
|---|---|---|---|
| Anthropic | Web search; documents and Web Fetch | Web search: always. Documents and Web Fetch: Citations() | Whole text part (Anthropic cites whole blocks) |
| Gemini API | Google Search, file search, web fetch | Turn on the tool | The supported text |
| Vertex AI | Google Search | Turn on the tool | The supported text |
| OpenAI Chat and Responses | Web pages; files on Responses | Turn on the tool | The link the model wrote; none for files |
When Citations() asks Anthropic for citations. Citations only attach to text. Anthropic writes its <cite> markup into anything that isn't text, so:
| Situation | What Citations() does | Evidence |
|---|---|---|
| Text output | Asks for citations | — |
Structured output, including [str, Model] (tool output) | Doesn't ask: clean fields, no document citations | Markup in ToolOutput fields 4 of 4 live; NativeOutput gets a 400 |
A function tool forced with tool_choice, in a request with a document or Web Fetch | Raises UserError before that request is sent. If a capability forces a tool on a later step, the run fails at that step | Markup in the tool's arguments 3 of 3 on Haiku 4.5 and Sonnet 4.5, 0 of 3 without |
| A forced tool with no document or Web Fetch in the request | Nothing: the setting doesn't change such a request, so it's allowed | — |
| Function tools the model chooses | Asks for citations; the docs warn, and suggest leaving Citations() off for such agents on smaller models | Markup in arguments 11 of 19 on Haiku 4.5 when they summarise the document, 0 of 7 on Sonnet 4.5, never without |
| OpenAI, Gemini | Nothing: these always cite search results | Structured output behaves the same with or without it (live, 2026-10-23) |
Nuances to know across the stack
These are the behaviours a user can be surprised by. Each is in the docs.
- Citations only exist on text. An answer returned through tool output (the default structured output, and also
[str, Model]) never has them, on any provider: tool call arguments can't carry citations. - On Anthropic and Bedrock,
Citations()is off for structured output, and an error with a forced tool when there's something to cite (tables here and in #7750), because the model writes<cite>markup into anything that isn't text. With tools the model chooses, smaller models put the markup into tool arguments about half the time when the arguments summarise a document. This is the one place whereCitations()can still corrupt data; worth a decision on whether a warning is enough. - Anthropic always cites web search, with or without
Citations(), so with web search its markup can reach structured output fields. That happens on main today. - On xAI,
Citations()changes the answer text (the model writes[[1]](...)markers), and is off for structured output (#7750). - Citations can arrive after their text when streaming: at the end of each part on Anthropic, Bedrock and OpenAI Responses, at the end of the response on Google, OpenAI Chat and xAI. Read them from the final response.
- Only some citations go back in history (#7752): Anthropic web search and OpenAI Responses url/file citations, to the same provider, while the text is unchanged. Document citations go back as plain text, because their document can't be identified once history changes.
- UI adapters: Vercel keeps citations through the frontend (#9639); AG-UI drops them.
Decisions
- No anchor rather than a wrong one. An anchor is set only when its position can be checked against the text: out-of-range offsets, and Google offsets that don't select Google's own segment text, get none.
- One small shape; the rest stays raw. The citation types hold only what every provider can fill in. Anthropic's document location and
encrypted_index, Google's confidence scores and OpenAI's file position stay inprovider_details['citations'], withprovider_nameset. - The setting is
citationsand covers Web Fetch too (agreed with Douwe).WebFetchTool.enable_citationsstays abool; Web Fetch is cited when either is on. - All of a request's Anthropic documents get citations, including tool-returned ones, because Anthropic requires all or none.
- Text output only, and an error for forced tools (table above). Silently corrupted fields or tool arguments are worse than no citations, and a forced tool gets the markup every time, so that combination fails loudly instead of quietly turning citations off. The error only fires when the request has a document or Web Fetch tool, since otherwise the setting changes nothing.
- Gemini offsets are repaired, not trusted. When Gemini splits an answer into several parts its offsets count across the joined parts, and they're sometimes one byte off (about 1 response in 10 live). Anchors are matched against the joined text, and fall back to the segment's text when it appears exactly once.
- Web Fetch citations get their page URL when the cited text matches a page fetched in this or an earlier response, and only when exactly one page matches: pages with different URLs and the same text at that range leave it a document source rather than guess. A fetched PDF stays a document source.
- Streaming sends a part's citations once its text is complete, in one
TextPartDeltathat sets bothcitationsandprovider_details['citations']. Google and OpenAI Chat only return them at the end of the response. - Unknown or empty citations are skipped, never guessed: unknown SDK types, OpenAI annotations that don't validate, and file citations that name no file.
- A streamed OpenAI Responses message gives one
TextPartper content part, like the non-streamed response. openai_include_raw_annotationsmoved toOpenAIChatModelSettings, which Responses settings extend, so it also works for OpenAI Chat and OpenRouter.
Limits
- Structured output on Anthropic gets no document citations (above). Anthropic always cites web search, so with web search its markup can still reach structured output fields; that already happens on main and is documented.
- OpenAI only attaches citations to text: a message with no text gets no part. Citations are left off when the text changed after it arrived, such as by removing thinking tags.
- Anthropic citations of client-provided search results (
search_result_location) aren't mapped. - The AG-UI adapter doesn't send citations to the frontend; the docs say so.
- Bedrock, OpenRouter and xAI are #7750, sending citations back in history is #7752, and the Vercel adapter is #9639.
Verification
- Recorded tests for Anthropic, Gemini, Vertex AI, OpenAI Responses and OpenAI Chat, all but the Gemini API also streamed. Anthropic Web Fetch is recorded streamed and not, and a recording with a document in the prompt and another returned by a tool shows Anthropic cites both.
- Every recorded Gemini anchor is checked against Google's segment text; a unit test covers an offset inside a multi-byte character.
- Live acceptance on 2026-10-22 and 2026-10-23: every anchor, source and raw entry compared with the provider's raw response, streamed and not, including Japanese and emoji text; then every output mode (text, tool, native, prompted) with
Citations()on and off on every provider, and function tools with documents. The structured-output rule, the forced-tool error and the earlier-fetch and Gemini offset fixes came from these runs.
Checklist
- Any AI generated code has been reviewed line-by-line by the human PR author, who stands by it.
- No breaking changes in accordance with the version policy.
- Any permitted compatibility impact has a label, warning, migration, release note, and exact API-check waiver.
- PR title is fit for the release changelog.