Add structured citations to `TextPart`

Merged
AD
9d ago
+6355-189

Adds TextPart.citations, so an app can show which sources back which part of a model's answer. Anthropic, Gemini, Vertex AI and OpenAI (Chat and Responses, and so OpenRouter and other providers on the OpenAI Chat mapping) return citations in one shape, streamed or not. The response text is never changed. In this PR, requests only change when citations are asked for with the Citations() capability (or ModelSettings(citations=True), which it sets); #7752 also sends some citations back in history.

Citation(
    sources=[WebCitationSource(url='https://example.com', title='Example')],
    anchor=ContentCitationAnchor(start=0, end=34),  # part.content[0:34] is the supported claim
)

How it works, at a glance

provider response
  └─ model adapter maps each citation it can check
       └─ TextPart(content,                                   ◄── text unchanged
                   citations=[Citation(sources, anchor)],       ◄── same shape for every provider
                   provider_details={'citations': [raw entries]})  ◄── everything else the provider returned
ProviderWhat's citedHow to turn it onAnchor
AnthropicWeb search; documents and Web FetchWeb search: always. Documents and Web Fetch: Citations()Whole text part (Anthropic cites whole blocks)
Gemini APIGoogle Search, file search, web fetchTurn on the toolThe supported text
Vertex AIGoogle SearchTurn on the toolThe supported text
OpenAI Chat and ResponsesWeb pages; files on ResponsesTurn on the toolThe link the model wrote; none for files

When Citations() asks Anthropic for citations. Citations only attach to text. Anthropic writes its <cite> markup into anything that isn't text, so:

SituationWhat Citations() doesEvidence
Text outputAsks for citations—
Structured output, including [str, Model] (tool output)Doesn't ask: clean fields, no document citationsMarkup in ToolOutput fields 4 of 4 live; NativeOutput gets a 400
A function tool forced with tool_choice, in a request with a document or Web FetchRaises UserError before that request is sent. If a capability forces a tool on a later step, the run fails at that stepMarkup in the tool's arguments 3 of 3 on Haiku 4.5 and Sonnet 4.5, 0 of 3 without
A forced tool with no document or Web Fetch in the requestNothing: the setting doesn't change such a request, so it's allowed—
Function tools the model choosesAsks for citations; the docs warn, and suggest leaving Citations() off for such agents on smaller modelsMarkup in arguments 11 of 19 on Haiku 4.5 when they summarise the document, 0 of 7 on Sonnet 4.5, never without
OpenAI, GeminiNothing: these always cite search resultsStructured output behaves the same with or without it (live, 2026-10-23)

Nuances to know across the stack

These are the behaviours a user can be surprised by. Each is in the docs.

  1. Citations only exist on text. An answer returned through tool output (the default structured output, and also [str, Model]) never has them, on any provider: tool call arguments can't carry citations.
  2. On Anthropic and Bedrock, Citations() is off for structured output, and an error with a forced tool when there's something to cite (tables here and in #7750), because the model writes <cite> markup into anything that isn't text. With tools the model chooses, smaller models put the markup into tool arguments about half the time when the arguments summarise a document. This is the one place where Citations() can still corrupt data; worth a decision on whether a warning is enough.
  3. Anthropic always cites web search, with or without Citations(), so with web search its markup can reach structured output fields. That happens on main today.
  4. On xAI, Citations() changes the answer text (the model writes [[1]](...) markers), and is off for structured output (#7750).
  5. Citations can arrive after their text when streaming: at the end of each part on Anthropic, Bedrock and OpenAI Responses, at the end of the response on Google, OpenAI Chat and xAI. Read them from the final response.
  6. Only some citations go back in history (#7752): Anthropic web search and OpenAI Responses url/file citations, to the same provider, while the text is unchanged. Document citations go back as plain text, because their document can't be identified once history changes.
  7. UI adapters: Vercel keeps citations through the frontend (#9639); AG-UI drops them.

Decisions

  • No anchor rather than a wrong one. An anchor is set only when its position can be checked against the text: out-of-range offsets, and Google offsets that don't select Google's own segment text, get none.
  • One small shape; the rest stays raw. The citation types hold only what every provider can fill in. Anthropic's document location and encrypted_index, Google's confidence scores and OpenAI's file position stay in provider_details['citations'], with provider_name set.
  • The setting is citations and covers Web Fetch too (agreed with Douwe). WebFetchTool.enable_citations stays a bool; Web Fetch is cited when either is on.
  • All of a request's Anthropic documents get citations, including tool-returned ones, because Anthropic requires all or none.
  • Text output only, and an error for forced tools (table above). Silently corrupted fields or tool arguments are worse than no citations, and a forced tool gets the markup every time, so that combination fails loudly instead of quietly turning citations off. The error only fires when the request has a document or Web Fetch tool, since otherwise the setting changes nothing.
  • Gemini offsets are repaired, not trusted. When Gemini splits an answer into several parts its offsets count across the joined parts, and they're sometimes one byte off (about 1 response in 10 live). Anchors are matched against the joined text, and fall back to the segment's text when it appears exactly once.
  • Web Fetch citations get their page URL when the cited text matches a page fetched in this or an earlier response, and only when exactly one page matches: pages with different URLs and the same text at that range leave it a document source rather than guess. A fetched PDF stays a document source.
  • Streaming sends a part's citations once its text is complete, in one TextPartDelta that sets both citations and provider_details['citations']. Google and OpenAI Chat only return them at the end of the response.
  • Unknown or empty citations are skipped, never guessed: unknown SDK types, OpenAI annotations that don't validate, and file citations that name no file.
  • A streamed OpenAI Responses message gives one TextPart per content part, like the non-streamed response.
  • openai_include_raw_annotations moved to OpenAIChatModelSettings, which Responses settings extend, so it also works for OpenAI Chat and OpenRouter.

Limits

  • Structured output on Anthropic gets no document citations (above). Anthropic always cites web search, so with web search its markup can still reach structured output fields; that already happens on main and is documented.
  • OpenAI only attaches citations to text: a message with no text gets no part. Citations are left off when the text changed after it arrived, such as by removing thinking tags.
  • Anthropic citations of client-provided search results (search_result_location) aren't mapped.
  • The AG-UI adapter doesn't send citations to the frontend; the docs say so.
  • Bedrock, OpenRouter and xAI are #7750, sending citations back in history is #7752, and the Vercel adapter is #9639.

Verification

  • Recorded tests for Anthropic, Gemini, Vertex AI, OpenAI Responses and OpenAI Chat, all but the Gemini API also streamed. Anthropic Web Fetch is recorded streamed and not, and a recording with a document in the prompt and another returned by a tool shows Anthropic cites both.
  • Every recorded Gemini anchor is checked against Google's segment text; a unit test covers an offset inside a multi-byte character.
  • Live acceptance on 2026-10-22 and 2026-10-23: every anchor, source and raw entry compared with the provider's raw response, streamed and not, including Japanese and emoji text; then every output mode (text, tool, native, prompted) with Citations() on and off on every provider, and function tools with documents. The structured-output rule, the forced-tool error and the earlier-fetch and Gemini offset fixes came from these runs.

Checklist

  • Any AI generated code has been reviewed line-by-line by the human PR author, who stands by it.
  • No breaking changes in accordance with the version policy.
  • Any permitted compatibility impact has a label, warning, migration, release note, and exact API-check waiver.
  • PR title is fit for the release changelog.