perf: avoid full HTML rendering for text-only CSV and TSV output
Summary
CSV and TSV partitioning still render and parse the entire HTML table when infer_table_structure=False, even though that HTML is then discarded. Large ordinary tables therefore pay for the HTML string, parsed tree, and compacted serialization while requesting only table text.
Generate normalized body text directly for that output mode. Render only the header, when requested, to preserve Pandas' label formatting. Frames containing missing values and text containing characters incompatible with the existing XML serialization use the original renderer, preserving its current formatting and error behavior.
Validation
- 36 compatibility/control cases compare complete serialized output with and without HTML, including element IDs and metadata. They cover both formats and header modes, literal HTML/entities, Unicode, quoted newlines/tabs, duplicate and blank headers, missing cells, NUL/control characters, header-only and empty files, and the existing XML-character error.
- The new cases pass on both the base revision and this change. The new cases plus the existing CSV/TSV suites report 172 passed, 9 skipped using normal package imports on Python 3.12.14. The nine skips require the optional
tiktokendependency. - Ruff 0.17.5
check,format --check, andgit diff --checkpass. The full repository suite was not run.
Measured workload
Same-host comparison of base ac932052e6e0ba3feab8a6e96b6d4480218e009f against this implementation: 20,000 data rows plus the first row, eight string columns, infer_table_structure=False, and languages=["eng"]. Each case has three paired fresh-process repetitions, alternating execution order, for 24 processes total. All input hashes and complete serialized-output hashes match within each case.
| Format | Include header | Median partition time, base → change | Median process peak RSS, base → change |
|---|---|---|---|
| CSV | false | 2.337 s → 0.146 s | 210,624 → 111,544 KiB |
| CSV | true | 2.260 s → 0.145 s | 210,312 → 112,080 KiB |
| TSV | false | 2.217 s → 0.141 s | 210,540 → 111,528 KiB |
| TSV | true | 3.569 s → 0.156 s | 210,400 → 111,856 KiB |
Peak RSS is measured with resource.getrusage(...).ru_maxrss and includes imports, input generation, partitioning, and serialization. These are results for the specified ordinary-table workload; missing-cell and incompatible-character fallbacks still use the original renderer. Timing varies on this shared host; the per-case peak-RSS reduction was 46.7–47.0%.
Checklist
- The PR title is a user-facing release-note entry with a conventional commit prefix.
- Apply a category label (
enhancement,bug, or an appropriate existing label), or ask a maintainer to apply it. Please applyenhancement. - For breaking changes, use
!in the title and explain impact and migration above. This change preserves the existing output and public signatures.