perf: skip discarded document detection in per-element language mode

AF
10h ago
+104-7

Summary

With detect_language_per_element=True, apply_lang_metadata() first joins all element text and detects the language of that full document, then discards that result and detects each element separately. This adds an unnecessary document allocation and detector invocation. For a short document, a custom language fallback also receives an extra combined-text call that does not contribute to any element.

Only build and detect the combined sample in document mode. Per-element mode now processes its individual texts directly, while preserving element identity, output order, automatic/explicit language selection and disabled-detection behavior.

Validation

  • The 13 new cases produce 10 failures and 3 passing controls on the original source. They wrap the real detector to observe its inputs and verify real output metadata; the fallback test checks its actual callback calls.
  • The final source plus existing language, empty-selection and text-partitioner suites reports 189 passed, 4 skipped on Python 3.12. The four skips are existing optional-backend cases.
  • With 2,000 short elements after detector warmup, median time over five calls decreases from 14.424 ms to 2.482 ms. Peak tracemalloc allocations decrease from 714,577 to 34,246 bytes. The language-output SHA-256 is identical. These measurements describe that workload and runner.
  • Repository-configured Ruff check, format check and source-integrity checks pass.
  • Validation run, exact benchmark, commands and JUnit artifacts.

Checklist

  • User-facing conventional-commit title.
  • Maintainer: please apply the performance category label.