perf: render the PDF once in hi_res instead of once per pipeline stage

Draft
CY
1mo ago
+481-201

What

hi_res renders the PDF once and shares the page images between layout inference, OCR and image/table extraction. save_elements also keeps one page image open instead of decoding the full-page PNG again for every element.

render calls, 4-page PDF, extract Image + Table   before: 16   after: 4
render calls, N pages, plain hi_res               before: 3N   after: N

Output is unchanged: the element list hashes identically before and after on every benchmark document.

Why

Every stage that needed page pixels rendered the whole PDF itself, so a request paid for the same render 3-4 times, and save_elements paid it once per extracted element type.

Relationship to the inference PR

Independent to merge. The single render only turns on when the installed unstructured-inference accepts pre-rendered pages (https://github.com/Unstructured-IO/unstructured-inference/pull/524, 1.6.14). With older versions hi_res renders per stage exactly as before, so this PR is a no-op until that release is picked up.

Measured

Warm in-process partition_pdf(strategy="hi_res", infer_table_structure=True) on an SND pod running the core-product-api image (4 CPU), main vs this branch, runs alternated A/B/B/A. Median wall seconds:

documentmodemainthis PRfasterpage renders
layout-parser-paper (16p)plain67.164.34%32 → 16
layout-parser-paper (16p)+ Image/Table extraction75.266.212%64 → 16
multi-column (13p)plain74.371.64%26 → 13
multi-column (13p)+ Image/Table extraction80.571.012%52 → 13
100page (first 40p)plain235.8229.13%80 → 40
100page (first 40p)+ Image/Table extraction302.1229.924%160 → 40

Peak RSS is unchanged (within 2%) and the element list hashes identically in every case.

Plain hi_res gains little because inference dominates; the win is on requests that extract images or tables, where each type previously re-rendered the PDF and re-decoded the page per element.

Validation

New unit tests for each hand-off. test_pdf_image passes except 5 Tesseract Korean tests that fail without the local kor traineddata. cubic: no issues. Tested against released inference 1.6.12 (no-op path) and the inference PR branch (single render).