fix(documents): rescue dollar-delimited math in AI-written HTML

Draft
PO
1mo ago
+238-42

Problem

  • Study notes are the product's core output, and the AI wrote math as plain text on the only path that saves them — a math-and-science app getting formulas wrong at the thing users came for.
  • parseDocumentAiHtml rescues <sub>/<sup> and h5/h6, but nothing repairs math. Math the model writes as $…$ or $$…$$ is stored as literal text; only display equations already in their own math block survived.
  • The tool prompt already tells the model to use Tiptap math markup and avoid dollar signs, but that is prompt-only. Wording was tightened once before and did not hold.

Changes

  • Mechanism: a deterministic rescue in rewriteLossyElements, beside the existing sub/sup handling, converts dollar-delimited math into the math nodes the schema renders.
Input in model HTMLResult
$E = mc^2$inline math node (<span data-type="inline-math">)
$$\int_0^1 x^2 dx$$block math node (<div data-type="block-math">)
$5 and $10, proseleft as text (mathLikeInlineBody guards money)
math inside <code>, <pre>, widget sourceleft as text
  • An inline pair converts only when its body reads as math; a $$…$$ pair is display math with no money reading, so a non-empty body is enough.
  • Moved the mathLikeInlineBody heuristic to a neutral module (src/lib/math-like-text.ts) shared by the chat Markdown normalizer and the document parser, so the document model no longer imports from the chat UI folder.
  • Flashcards and quizzes parse through the same parseDocumentAiHtml, so they get the rescue too.

Testing

  • pnpm check (format, lint, types) — pass.
  • New unit tests for inline/block rescue, money left alone, and code/widget text kept literal.
  • pnpm vitest --run over documents, content, flashcards, quizzes — 81 pass.

Closes #837


Created with PostHog Desktop from this inbox report, addressing #837.


View with [code]smith Autofix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need. Autofix is disabled.