fix(xlsx): keep cell text like N/A and None instead of emptying it

Merged
WE
10d ago
+194-2

Problem

partition_xlsx() reads each worksheet with pd.read_excel() and the pandas default NA strings, so a cell whose text is N/A, NA, null, None, NaN, #N/A or similar is read as missing and comes out empty. The missing cells also change how the worksheet is split into subtables, so a whole table can fall apart:

import io, openpyxl
from unstructured.partition.xlsx import partition_xlsx

wb = openpyxl.Workbook()
for row in (["Item", "Status"], ["Pear", "N/A"], ["Fig", "NA"], ["Plum", "null"], ["Kiwi", "None"]):
    wb.active.append(row)
f = io.BytesIO()
wb.save(f)

[(type(e).__name__, e.text) for e in partition_xlsx(file=f)]
# before: [('Table', 'Item Status'), ('Title', 'Pear'), ('Title', 'Fig'), ('Title', 'Plum'), ('Title', 'Kiwi')]
# after:  [('Table', 'Item Status Pear N/A Fig NA Plum null Kiwi None')]

Fix

Worksheets are read with keep_default_na=False, na_values=[""], so only empty cells are missing. Empty cells still come back as NaN, which keeps subtable detection and the empty cells in text_as_html unchanged (test_partition_xlsx_subtables covers the splitting). None of the XLSX files in example-docs contain NA-like text, so their outputs do not change.

#4373 makes the equivalent change for CSV and TSV; this PR only touches the XLSX reader.

Testing

pytest test_unstructured/partition/test_xlsx.py, including a new regression test built from the example above.