perf: render the PDF once in hi_res instead of once per pipeline stage
What
hi_res renders the PDF once and shares the page images between layout inference, OCR and image/table extraction. save_elements also keeps one page image open instead of decoding the full-page PNG again for every element.
render calls, 4-page PDF, extract Image + Table before: 16 after: 4
render calls, N pages, plain hi_res before: 3N after: N
Output is unchanged: the element list hashes identically before and after on every benchmark document.
Why
Every stage that needed page pixels rendered the whole PDF itself, so a request paid for the same render 3-4 times, and save_elements paid it once per extracted element type.
Relationship to the inference PR
Independent to merge. The single render only turns on when the installed unstructured-inference accepts pre-rendered pages (https://github.com/Unstructured-IO/unstructured-inference/pull/524, 1.6.14). With older versions hi_res renders per stage exactly as before, so this PR is a no-op until that release is picked up.
Measured
Warm in-process partition_pdf(strategy="hi_res", infer_table_structure=True) on an SND pod running the core-product-api image (4 CPU), main vs this branch, runs alternated A/B/B/A. Median wall seconds:
| document | mode | main | this PR | faster | page renders |
|---|---|---|---|---|---|
| layout-parser-paper (16p) | plain | 67.1 | 64.3 | 4% | 32 → 16 |
| layout-parser-paper (16p) | + Image/Table extraction | 75.2 | 66.2 | 12% | 64 → 16 |
| multi-column (13p) | plain | 74.3 | 71.6 | 4% | 26 → 13 |
| multi-column (13p) | + Image/Table extraction | 80.5 | 71.0 | 12% | 52 → 13 |
| 100page (first 40p) | plain | 235.8 | 229.1 | 3% | 80 → 40 |
| 100page (first 40p) | + Image/Table extraction | 302.1 | 229.9 | 24% | 160 → 40 |
Peak RSS is unchanged (within 2%) and the element list hashes identically in every case.
Plain hi_res gains little because inference dominates; the win is on requests that extract images or tables, where each type previously re-rendered the PDF and re-decoded the page per element.
Validation
New unit tests for each hand-off. test_pdf_image passes except 5 Tesseract Korean tests that fail without the local kor traineddata. cubic: no issues. Tested against released inference 1.6.12 (no-op path) and the inference PR branch (single render).