Stop a note under a recovered table row being glued into that row

Merged
BI
1mo ago
+148-0

User description

Summary

  • Before: when the PDF reader recovers a table row that find_tables cut off, any short line just under that row was merged into one of its cells. A note such as "See note" under "Clozapine" became "Clozapine See note" in the table, the accessible table text and the table facts.
  • After: a lone line joins the row above only when that cell actually wrapped. The cell's text must reach closer to the column's right edge than the width of the next line's first word. Otherwise the line is treated as a note under the table and left out.
  • Ledger #ZWJCK4, rechecked on current main: its main complaint (prose admitted by position alone) was already fixed in #3571 and is pinned by tests/pdf-table-extraction.test.ts. This PR fixes the leftover. The update request downgrades it to P2 and records what is still open: text laid out in the table's own columns directly beneath it (for example an abbreviation key) can still become a row when a drawing extends the crop over it.

Areas touched: Source intake and indexing, Knowledge and records
RAG impact: no retrieval behaviour change — extractor output only changes for documents ingested or re-indexed after merge; live chunks, embeddings and ranking are untouched until then.

Verification

  • Checks run:
    • New test "keeps a short note under a recovered row out of that row's cells" fails with the fix removed (Tests 1 failed | 4 passed (5)) and passes with it.
    • npx vitest run tests/pdf-table-extraction.test.ts tests/pdf-extractor.test.ts: Tests 10 passed | 1 skipped (11).
    • python3 -m unittest discover -p "test_extract_pdf_assets*.py" in worker/python: Ran 53 tests ... OK.
    • npm run test:focused, eslint --max-warnings 0, docs:check-links passed.

Risk and rollout

  • Risk: low. One merge rule in the edge-strip row recovery. A genuinely wrapped cell still joins (tested). A wrapped cell in a column whose recorded edge sits well right of the real wrap point could now stay split as a dropped line.
  • Rollback: revert this PR.
  • Provider or production effects: none on merge. Takes effect for new uploads or a re-index.

Notes

  • Draft until the owner marks it ready.

Generated by Claude Code

Summary by cubic

Stops a short note under a recovered table row being merged into that row's cells. A lone line now joins the row above only when that cell actually wrapped (free space is counted on both sides, covering right-aligned and centred cells) or when the line starts at the cell's left edge with a bullet, lower-case word, or number (a dose under a drug name); otherwise it stays out of the table.

Bug Fixes

  • Wrapped cells, deliberate line breaks, and doses under a drug name still join; a lone line also fills a still-empty cell even when it starts on a lower staggered baseline, as it did before. Notes under short right-aligned values stay out, as do star and numbered footnotes such as * Off-label use and 1 Not PBS listed. Tests in tests/pdf-table-extraction.test.ts cover all cases.
  • Updates ledger #ZWJCK4 to P2; its original fault was already fixed in #3571, this fixes the leftover.
  • Still open: text laid out in the table's own columns directly beneath it can become a row when a drawing extends the crop over it.
  • Output changes only for documents ingested or re-indexed after merge.

CodeAnt-AI Description

Keep notes and footnotes out of recovered PDF table rows

What Changed

  • A line below a recovered table row is no longer added to a cell just because it is nearby; it joins only when it appears to continue that cell’s text.
  • Wrapped text, doses, and bulleted instructions remain in their cells, while short notes and footnotes stay outside the row.

Impact

✅ Fewer notes shown inside PDF table cells
✅ Preserved multi-line doses and instructions in extracted tables
✅ Cleaner table text in newly indexed PDFs

💡 Usage Guide

Checking Your Pull Request

Every time you make a pull request, our system automatically looks through it. We check for security issues, mistakes in how you're setting up your infrastructure, and common code problems. We do this to make sure your changes are solid and won't cause any trouble later.

Talking to CodeAnt AI

Got a question or need a hand with something in your pull request? You can easily get in touch with CodeAnt AI right here. Just type the following in a comment on your pull request, and replace "Your question here" with whatever you want to ask:

@codeant-ai ask: Your question here

This lets you have a chat with CodeAnt AI about your pull request, making it easier to understand and improve your code.

Example

@codeant-ai ask: Can you suggest a safer alternative to storing this secret?

Preserve Org Learnings with CodeAnt

You can record team preferences so CodeAnt AI applies them in future reviews. Reply directly to the specific CodeAnt AI suggestion (in the same thread) and replace "Your feedback here" with your input:

@codeant-ai: Your feedback here

This helps CodeAnt AI learn and adapt to your team's coding style and standards.

Example

@codeant-ai: Do not flag unused imports.

Retrigger review

Ask CodeAnt AI to review the PR again, by typing:

@codeant-ai: review

Check Your Repository Health

To analyze the health of your code repository, visit our dashboard at https://app.codeant.ai. This tool helps you identify potential issues and areas for improvement in your codebase, ensuring your repository maintains high standards of code health.