fix: keep conversation notes when the model leaves quotes unescaped
What changed and why
Conversation notes failed closed when the structure model left prose quotes unescaped inside an otherwise complete JSON object. JsonOutputParser raised OutputParserException from langchain_core/output_parsers/json.py:91, and _get_structured turned that into a processing failure, so the conversation never got a summary.
Error group (source: gcp):
{"source": "gcp", "service": "backend", "group": "json.py:91 | langchain_core.exceptions.OutputParserException: Invalid json output: {"title": "Fish identification", "overvi", "count": 1, "id": "gcp:backend-json-py-91-langchain-core-exceptions-outputparserexception-invalid-json-output-title-fish-identificati"}
Root cause
The traceback is process_conversation.py _get_structured → conversation_processing.py _get_shaped_conversation_notes → extraction_parser.parse. The completion is a StructuredExtraction object (title, overview, sections, …) whose string values contain raw " characters. Langchain's partial JSON repair cannot recover that shape (Expecting ',' delimiter immediately after the interior quote), so the strict parser raises and the notes call fails.
Fix
_parse_notes_model_output still parses with PydanticOutputParser. Only after that raises does it escape interior quotes whose next non-space character is not a JSON closer (, } ] : "), then parse again. Valid JSON, already-escaped quotes, and a missing comma between fields are unchanged.
Product invariants affected
none
How it was verified
python3 -m py_compile backend/utils/llm/conversation_processing.py backend/tests/unit/test_conversation_notes_json_quotes.py
# COMPILE_OK
backend/.venv python -m pytest tests/unit/test_conversation_notes_json_quotes.py -q --tb=short
# .... [100%]
# 4 passed in 3.72s
The logged Fish identification completion still raises on the strict parser and now parses through _parse_notes_model_output as StructuredExtraction with that title, a non-empty overview, and one section.
Tests
backend/tests/unit/test_conversation_notes_json_quotes.py covers unescaped quotes in overview and in a section body, already-escaped quotes, and a missing comma that must still raise.
UI evidence
UI-Evidence: none -- backend notes parser only
Failure class (fixes)
Failure-Class: none