fix(md): recognize ATX headings indented by up to three spaces

CI
1d ago
+43-1

Summary

partition_md left a heading as plain text when it was indented by one to three spaces, for example " ## Chapter 1". CommonMark allows this indentation. The heading came out as UncategorizedText with the literal ## in it, so no Title element was created and the following elements got the wrong parent_id (and no category_depth). This is the case reported in #3952, where the sample string is built with \ line continuations that add a leading space to each line.

Root cause: Python-Markdown's HashHeaderProcessor only matches # in the first column. The fix adds a small internal extension that registers a subclass of that processor. Its regex allows [ ]{0,3} before the #. Four or more spaces still give an indented code block. Fenced code is stashed by the preprocessor before block parsing, so it is not affected. The extension is always added after the user's extensions list.

Fixes #3952

Validation

  • New tests in test_unstructured/partition/test_md.py:
    • indented headings (1 and 3 spaces) become Title elements with the right category_depth and parent_id chain (fails on main, passes now);
    • a 4-space-indented # ... line is still code, not a heading.
  • .venv/bin/python -m pytest test_unstructured/partition/test_md.py -> 43 passed
  • ruff check and ruff format --check on the changed files: clean.
  • I did not run the full suite (limited disk on my machine).

Repro from the issue now gives Title(Chapter 1) -> Title(Section) -> body text, each with the expected parent.