Add instrumentation version 6, emitting tool results under `role: 'tool'`

Merged
DS
2mo ago
+127-11

This pull request was posted by Claude Code using claude-opus-5 on behalf of David.

David has not reviewed the code. He reviewed the two design decisions — version 6 being non-default, and the tool-availability delta keeping system — and directed the rest.

Why we make these changes

messages_to_otel_messages grouped a ModelRequest's parts by isinstance(p, SystemPromptPart | ToolAvailabilityDeltaPart) and labelled the result 'system' if is_system else 'user'. A tool result therefore arrived as a tool_call_response part inside a role: "user" message, while the GenAI semantic conventions pair that part with role: "tool". Role had no 'tool' member, so the right role was not representable. The reporter exports these spans to Databricks/MLflow, where the message role is what their consumers key on.

This is a changed value on telemetry we already emit, not an addition, so it ships as instrumentation format version 6 and versions 2–5 keep emitting user. That follows #4661, which took a new version to flip a span status on already-emitted telemetry while shipping its purely-additive half to every version in the same PR. Version 6 is not the default — #4219 and #4661 both shipped non-default, and the default flips at a major (TODO(v3) marker added next to the existing one).

The issue's second symptom — ToolAvailabilityDeltaPart appearing as a mid-conversation system message — is not changed, on any version. The semconv role vocabulary is exactly system | user | assistant | tool and none of those means "the set of tools changed"; tool is the one that would actively mislead, being the role paired with tool_call_response. system is also the mapping #6793 chose deliberately when it introduced the part. The reasoning is recorded in _otel_message_role's docstring so it doesn't get re-reported.

New public surface

  • InstrumentationSettings(version=6) — the only new value; version widens to Literal[2, 3, 4, 5, 6].

_otel_messages.Role gains 'tool' unconditionally rather than per version: it is a private type describing what the field can hold, and only the mapping is gated.

Goal — a tool result carries the role the semantic conventions give it

Before — one user message holding everything the request sent:

InstrumentationSettings(include_content=True).messages_to_otel_messages(history)
# -> [{'role': 'system',    'parts': [text]},
#     {'role': 'user',      'parts': [text]},
#     {'role': 'assistant', 'parts': [tool_call]},
#     {'role': 'user',      'parts': [tool_call_response, tool_call_response, text]}]

After, on version 6 — the request splits by role, in part order:

InstrumentationSettings(include_content=True, version=6).messages_to_otel_messages(history)
# -> [{'role': 'system',    'parts': [text]},
#     {'role': 'user',      'parts': [text]},
#     {'role': 'assistant', 'parts': [tool_call]},
#     {'role': 'tool',      'parts': [tool_call_response, tool_call_response]},
#     {'role': 'user',      'parts': [text]}]

A retry that answers a tool call goes to tool with the returns; a retry that answers nothing stays user, which is the role it reaches the model as — openai.py maps exactly this split to role='tool' / role='user' on the wire.

Playground 
from pydantic_ai.messages import (
    ModelRequest, ModelResponse, RetryPromptPart, SystemPromptPart,
    ToolCallPart, ToolReturnPart, UserPromptPart,
)
from pydantic_ai.models.instrumented import InstrumentationSettings

history = [
    ModelRequest(parts=[SystemPromptPart(content='Be brief.'), UserPromptPart(content='Convert 10 EUR.')]),
    ModelResponse(parts=[ToolCallPart(tool_name='convert', args={'amount': 10}, tool_call_id='call_1')]),
    ModelRequest(
        parts=[
            ToolReturnPart(tool_name='convert', content='11 USD', tool_call_id='call_1'),
            RetryPromptPart(content='Unknown currency.', tool_name='convert', tool_call_id='call_2'),
            RetryPromptPart(content='Output did not validate.'),
            UserPromptPart(content='Thanks.'),
        ]
    ),
]

for version in (5, 6):
    settings = InstrumentationSettings(include_content=True, version=version)
    print(version, [m['role'] for m in settings.messages_to_otel_messages(history)])
# 5 ['system', 'user', 'assistant', 'user']
# 6 ['system', 'user', 'assistant', 'tool', 'user']

Test: test_messages_to_otel_messages_request_roles_v6

What changes for existing users: nothing. Version 5 stays the default and its output is byte-identical — the same test pins ['system', 'user', 'assistant', 'user'] for the default alongside the version 6 snapshot, and every existing snapshot across tests/test_logfire.py, tests/test_include_binary_content.py, tests/test_messages.py, tests/test_history_processor.py and tests/models/test_snowflake.py is unchanged.

Checklist

  • Any AI generated code has been reviewed line-by-line by the human PR author, who stands by it.
  • No breaking changes in accordance with the version policy.
  • PR title is fit for the release changelog.