fix(cleaners): handle empty text in ordered-bullet cleaners

AS
2mo ago
+17-0

Summary

clean_ordered_bullets() and extract_ordered_bullets() read the first token of
text.split() without checking that a token exists, so empty or whitespace-only text
raises IndexError instead of passing through:

>>> from unstructured.cleaners.core import clean_ordered_bullets
>>> clean_ordered_bullets("")
IndexError: list index out of range

Every other cleaner in unstructured/cleaners/core.py returns '' for empty input —
clean_bullets, clean_extra_whitespace, clean_dashes, clean_trailing_punctuation,
clean_non_ascii_chars, clean_ligatures, replace_unicode_quotes,
remove_punctuation — so this looks like an oversight rather than an intended contract.

It matters because elements with empty text are common, and cleaners are applied per
element. A single empty element aborts the whole run:

from unstructured.documents.elements import Text
from unstructured.cleaners.core import clean_ordered_bullets

for el in [Text("1.1 Introduction"), Text(""), Text("2.1 Methods")]:
    el.apply(clean_ordered_bullets)   # raises IndexError on the empty one

Changes

Both functions now return early when text.split() is empty — the text unchanged for
clean_ordered_bullets, and (None, None, None) for extract_ordered_bullets, which is
what it already returns for any text that carries no bullet.

Testing

The snippet above runs clean on this branch. Or via the existing parametrized tests,
which now cover "", " " and "\n" for both functions:

pytest test_unstructured/cleaners/test_core.py test_unstructured/cleaners/test_extract.py
  • 133 passed on this branch (127 before, plus the 6 new cases)
  • The 6 new cases fail on main with IndexError, pass here
  • ruff check / ruff format --check clean on the changed files

CHANGELOG and __version__.py bumped to 0.25.2-dev1.

Written with AI assistance (Claude Code); I reviewed the change and ran the checks above.