[Feat]: tighten KB Indexed badge + folder cascade delete + pmid attribution

Closed
FI
3mo ago
+1283-227

Problem

  • KB "Indexed" badge reported success based on kb_chunk.status='success'
    alone, but pgvector hadn't received the embedding yet — the doc was
    vector-less when the badge said "Indexed".
  • kb_observability spans for concurrent KB uploads were getting tagged
    with the wrong parent_message_id (the next user message, not the
    trigger). Root cause was a shared instance field in
    CapturingHandler.
  • DELETE /api/kb/folders/[id] refused with 409 NON_EMPTY on any non-empty
    folder; no path for the user to clean up a whole folder.

What changed

fix(observability) — 9e39a9c

  • CapturingHandler.handleChainStart resolves parent_message_id per-call
    from metadata.parent_message_id (LangGraph-configurable passthrough,
    set per-run via runs.create({config:{configurable:{parent_message_id}}}))
    with lastHumanMessageId(inputs.messages) as fallback. Dropped the
    shared instance field.
  • findLatestParentMessageId now calls langGraphClient.threads.getState
    and parses state.values.messages via lastHumanMessageId — the
    observability_spans column is not read for backfill.

feat(kb) — feb73bb

  • successChunks redefined: counts chunks with
    status='success' AND embedding IS NOT NULL. New
    embeddingPendingChunks counts status='success' AND embedding IS NULL.
  • "Indexed" badge now requires vectors AND
    entityCount > 0 OR relationshipCount > 0 (hybrid-search third leg
    needs graph rows to score against). Two intermediate states:
    Extracting (X/Y) (OCR in flight) and Embedding (X/Y) (vector +
    graph extract in flight).
  • Surfaced entityCount + relationshipCount through 4 type files
    (components/tool-ui/kb/types.ts, components/settings/kb-view/types.ts,
    backend/tool/kb/types.ts, backend/tool/kb/list-documents.ts) and the
    ChunksStatusBadge prop list.

feat(kb) — a73a179

  • splitFileToPageNode + pageToMarkdownNode skip on retryFailed /
    retryFailedChunks (was chunksOnly only); prepareKBDataNode reads
    config.configurable.parent_message_id first.
  • DELETE /api/kb/folders/[id]?deleteAll=true cascades through every
    doc in the folder (chunks, embeddings, attachments). Settings dialog
    adds a switch + typed "delete all" confirmation.
  • DocDeleteDialog / DocReprocessDialog close themselves via
    onOpenChange(false) before firing onDeleted / onReprocessed;
    long titles get break-all. FolderSidebar callbacks no longer reset
    selection on submit (the dialog does it via onOpenChange).
  • ObservabilityPopover no longer auto-closes on row click — the Sheet
    is the singleton entry.
  • kb-mention popover: scrollable body + sticky header padding tweak.

fix(kb) — 9616f02

  • Badge elements get shrink-0 on icons + min-w-0 max-w-full on
    shell + truncate min-w-0 on label so the fixed-width badge column
    in DocRow doesn't push siblings off-grid on long status text.

Docs

  • docs/APIS.md: new indexing-count fields on GET /api/kb/documents
    • GET /api/kb/documents/[id], chunks[].hasEmbedding, folder
      ?deleteAll=true cascade.
  • docs/OBSERVABILITY.md: per-call pmid resolution from metadata +
    langGraphClient.threads.getState fallback.
  • docs/TOOLS.md: expanded list_documents payload shape shared with
    the Settings badges.

Tests

  • pnpm typecheck: clean
  • pnpm lint: clean
  • pnpm test: 131 files / 1367 passed
  • Added coverage: tests/api/kb/folder.test.ts (cascade delete) +
    tests/api/kb/reprocess.test.ts (retryFailedChunks pipeline skips
    OCR + preserves doc.status='success').

🤖 Generated with Claude Code


Summary by cubic

Tightened the Knowledge Base “Indexed” badge to require embeddings, added entity/relationship counts across APIs, enabled safe folder cascade delete (with attachment cleanup), and fixed parent_message_id attribution with a 500ms-bounded backfill. Also corrected OCR retry modes and applied small UI polish.

  • New Features

    • Badge and counts: “Indexed” now requires all chunks have embeddings; shows “Extracting (X/Y)” and “Embedding (X/Y)”. entityCount/relationshipCount are surfaced but not required for “Indexed”. Added embeddingPendingChunks, entityCount, and relationshipCount to GET /api/kb/documents, GET /api/kb/documents/[id], and list_documents; doc detail exposes chunks[].hasEmbedding.
    • Deletion and reprocess: DELETE /api/kb/folders/[id]?deleteAll=true cascades through docs, chunks, embeddings, attachments, and related threads; dialog adds a toggle and typed confirm. Reprocess modes: retryFailedChunks and chunksOnly skip OCR; retryFailed re-OCRs failed pages. prepareKBDataNode prefers config.configurable.parent_message_id.
  • Bug Fixes

    • Observability: resolve parent_message_id per call from metadata with a langGraphClient.threads.getState fallback capped at 500ms and fetched in parallel. Removed shared handler state to prevent cross-run clobbering.
    • Counts and UI: independent queries fix entityCount/relationshipCount inflation; tighter badge truncation and mixed-state logic (don’t mark as failed while embeddings are pending); ObservabilityPopover stays open on row click; kb-mention popover height fixed; dialogs close before callbacks.

Greptile Summary

The PR tightens KB indexing status, adds folder cascade deletion, corrects observability attribution, and repairs failed-page OCR retries.

  • Counts only successfully embedded chunks as indexed and exposes graph/indexing counts through APIs and UI.
  • Adds confirmed deletion of folders and their documents, including transactional attachment-row cleanup.
  • Resolves observability parent-message attribution per run rather than through shared handler state.
  • Reuses stored pages while rerunning OCR only where failed during retry processing.

Confidence Score: 5/5

The PR appears safe to merge.

No blocking failures remain in the fixes associated with the previous review threads.

Important Files Changed

FilenameOverview
lib/kb/queries.tsFolder and document deletion now remove associated attachment rows transactionally while preserving content-addressed R2 objects.
app/api/kb/documents/[id]/route.tsIndependent, user-scoped entity and relationship queries correctly avoid Cartesian count inflation.
backend/node/kb/page-to-markdown-node.tsFailed-page retries now rerun OCR while preserving successful cached pages.
backend/node/kb/split-file-page-node.tsReprocessing modes reuse pages loaded from persisted document state instead of splitting the source again.
components/tool-ui/kb/status-badge.tsxIndexing badges distinguish extraction, embedding, failure, and fully indexed states using embedding-aware chunk counts.
lib/observability/callback.tsParent-message attribution is resolved per invocation to avoid shared-state interference between concurrent runs.

Reviews (5): Last reviewed commit: "test(observability): mock langGraphClien..." | Re-trigger Greptile