fix: distinguish NewCM text fonts from math fonts
Related Issue(s)
Motivation and Context
The Stanford CS336 assignment PDF uses NewCM10-Regular, NewCM10-Bold, and NewCM10-Italic for ordinary prose, alongside NewCMMath-Book for mathematics. The precise formula-font patterns currently match every NewCM / NewComputerModern font, so the body text is classified as formulas and protected with placeholders rather than sent for translation.
This also explains why --formular-font-pattern ".*Math.*" cannot work around the issue: the precise built-in pattern returns before the custom pattern is evaluated.
Summary of Changes
In babeldoc/format/pdf/document_il/utils/formular_helper.py, narrow the two precise matches to NewCMMath and NewComputerModernMath. Ordinary NewCM fonts can then follow the existing text/custom-pattern classification, while actual math fonts remain protected.
In tests/test_formular_helper.py, cover regular/bold/italic text, PDF subset prefixes, both math-font spellings, existing Computer Modern/Latin Modern behavior, and page/XObject font scopes. Each case runs with the default formula pattern and the custom .*Math.* pattern.
#614 restores the paragraph bounding-box IoU fallback; this change addresses the separate font-classification cause documented in #613.
PR Type
- 🐛 Bug Fix
- 🧪 Test
Breaking Changes
None.
Testing Instructions
uv run pytest tests/test_formular_helper.py -q: 24 passed, importing the production module and using the real IL dataclasses.uv run ruff check .: passed.uv run ruff format --check babeldoc/format/pdf/document_il/utils/formular_helper.py tests/test_formular_helper.py: passed.git diff --check: passed.
PyMuPDF inspection of the linked PDF found 97,688 characters in NewCM10-Regular, 6,105 in bold, 884 in italic, and 2,674 in NewCMMath-Book, confirming that the affected font names occur in the reported document.
Contributor Checklist
- Read the contribution guide and kept the patch small.
- Reviewed the changed code.
- Linked the existing issue and added regression tests.
- The changed files follow the project's formatting and lint rules.
- No new dependency or user-facing configuration is introduced.
Limitations
Verification covers offline font classification, font-resource scopes, and the source PDF's font metadata. End-to-end translation through an external service is unverified.
Summary by cubic
Fixes #613: NewCM text fonts (e.g. NewCM10-Regular) were matched by the precise formula-font pattern and protected as math, preventing translation. The pattern now only matches math fonts (NewCMMath*, NewComputerModernMath*), so ordinary NewCM text fonts classify as text. Adds regression tests for font classification and page/XObject scopes, including the custom .*Math.* pattern.