perf(layout): Use linear memory when cleaning up large region groups

Merged
CY
1mo ago
+260-12

Summary

Layout cleanup for large connected groups of detected regions no longer builds several full N×N float arrays. It compares one candidate row at a time, so working memory grows linearly with group size instead of quadratically.

On a 4,000-region workload:

  • Peak RSS: 1,098.86 → 350.32 MiB.
  • Time: 0.641 → 0.273 s.
  • Output is identical.
  • Under a 768 MiB limit, main is OOM-killed and this branch completes.

Behavior

  • Unchanged: containment decisions, output order and every element attribute.
  • Worst-case comparison time is still quadratic. Only memory use changes.

Validation

  • The full non-slow unit suite, Ruff and formatting pass, and CI is green.
  • Regression tests cover:
    • exact parity with the dense reference implementation;
    • all element attributes;
    • degenerate, non-finite and extreme coordinates;
    • containment thresholds;
    • allocation bounds;
    • detection-pipeline routing.
  • Main and this branch were compared across 6,000 generated cases (six input dtypes, ten thresholds). Outputs and output dtypes are identical.
  • Isolated Linux containers, 2 CPU cores, equal memory limits, three alternating runs of a 4,000-region connected chain through PageLayout and detection-model deduplication:
    • Results are the numbers above, with identical full output hashes.
    • The detector is synthetic, so these figures cover layout processing, not end-to-end inference latency.
  • Built wheel:
    • The regression suite passes.
    • A real three-page YOLOX + OCR PDF partition produced nine identical elements on main and on this branch, with three calls through the changed function on each.

Release

Targets 1.6.14, the next patch release after main (1.6.13). This is independent of #521, which removes the grouping matrix in the preceding stage.

This PR can merge first. Before merging #518, #520 or #521, update its version and changelog to the next patch version on main at that time. Downstream pins need to adopt the published release. This fixes a reproduced OOM; it is not claimed to account for every production OOM.