fix(rag): drop stale legacy chunks from document summaries

Merged
BI
24d ago
+71-3

Summary

  • Before: a reindexed document could be summarised from both its old (legacy, NULL-generation) chunks and its new committed-generation chunks, because the fenced commit path leaves legacy rows in place and the summary query selected both.
  • After: legacy chunks are used only while the committed generation has written no chunks of its own, which matches the commit RPC's "preserve legacy rows unless this generation wrote replacements" rule. Documents that were never reindexed still summarise from their legacy chunks.
  • How: loadDocumentSummaryContext filters out NULL-generation rows whenever any committed-generation row is present. No query, migration or ranking change.
  • Follow-up to the two protected search findings deferred from #3652 and #3653, with the owner's approval to draft. The second finding (repeated rag_cache_scope_revision_v1 calls in rag-cache.ts) is deliberately not changed. Every call is a fresh before/after fence against revocation and mid-request reindexing, and the function's own contract forbids a memo. Making the RPC cheaper would need a database change.

Areas touched: Answer engine
RAG impact: no retrieval behaviour change — only the document-summary context changes; search retrieval, ranking, ordering and RPCs are untouched, and the retrieval eval canary does not exercise summaries

Verification

  • Checks run:
    • Both new tests in tests/rag-document-summary.test.ts were run without the fix: "drops stale NULL-generation chunks once the committed generation has rows" failed (Tests 1 failed | 5 passed (6)). With the fix: Tests 6 passed (6).
    • npm run test:focused: Tests 1328 passed (1328).
    • Typecheck passed, and eslint and prettier are clean on the changed files.

Risk and rollout

  • Risk: low. If a document's committed generation has some chunks, its summaries no longer include its legacy chunks. That is the intended behaviour.
  • Rollback: revert this single commit.
  • Provider or production effects: None.

Notes

  • Draft until the owner decides to merge.

Generated by Claude Code

Summary by cubic

Fixes document summaries mixing stale legacy chunks with newly committed chunks after a reindex.

loadDocumentSummaryContext now drops NULL-generation (legacy) rows whenever the committed generation has written chunks of its own, matching the commit RPC's preserve rule. Documents that were never reindexed still summarise from their legacy chunks.

  • Adds tests covering both the stale-chunk drop and the legacy-only fallback.
  • Search retrieval, ranking, and RPCs are unchanged.