Add instrumentation version 4 to match OTel GenAI semantic conventions for multimodal input
Merged
AD
9mo ago+570+570+012345678901234567890123456789-23-23-01234567890123456789
Adds support for OpenTelemetry GenAI semantic conventions for multimodal content via a new instrumentation version 4. This addresses the issue where image, audio, video, and document content was not following the official spec format.
Changes:
- Add UriPart and BlobPart TypedDicts to _otel_messages.py for the spec-compliant formats
- Add version 4 to InstrumentationSettings (versions 1-3 remain unchanged)
- Update messages.py to convert media URLs and binary content to the new format when version >= 4
- Apply version 4 format to both input messages (UserPromptPart) and output messages (ModelResponse/FilePart)
- Add comprehensive tests for version 4 multimodal inputs and outputs
- Update logfire.md documentation to explain version 4
Version 4 changes the format in gen_ai.input.messages and gen_ai.output.messages:
URL-based media (ImageUrl, AudioUrl, DocumentUrl, VideoUrl):
- Old (v1-3):
{"type": "image-url", "url": "..."} - New (v4):
{"type": "uri", "modality": "image", "uri": "...", "mime_type": "..."}
Binary content (BinaryContent in input, FilePart in output):
- Old (v1-3):
{"type": "binary", "media_type": "...", "content": "..."} - New (v4):
{"type": "blob", "modality": "image", "mime_type": "...", "content": "..."}
Privacy controls:
mime_typeis always included (metadata)uriandcontentare omitted wheninclude_content=False(sensitive data)
This is a non-breaking change as version 2 remains the default.
Fixes #4200