improvement(search): consolidate knowledge search around live Sim Search and document-decided workspace retrieval

Merged
WA
1mo ago
+7925-8851

Summary

  • Organization Sim Search is live-only. The indexed org-search implementation stays in the repo but is dormant: it lives entirely in lib/sim-search/indexed/ (and app/**/indexed/ for its UI) behind one gate, isIndexedOrgSearchEnabled(), and every indexed use case refuses on its own while dormant
  • Workspace knowledge base search (Knowledge block, v1/v2, MCP, Sim tools) decides access on the document for every principal. A search that names a search-index knowledge base keeps working the same way while indexed search is dormant. The per-row ACL machinery (access plan, reach/permitted sets, on-row walk, embedding_keyword_search, per-source walks) now only runs for search-index bases while indexed search is on. Signed-in workspace searches run fewer statements; live source proof stays lazy; actorless legs keep their prior (unbudgeted) behavior
  • Splits shared retrieval into candidates.ts, tag-filters.ts, and vector-leg.ts
  • Removes the knowledge-async-projection, knowledge-projection-fill, and knowledge-tin-keyword flags and their code paths
  • Less background DB work: no per-source HNSW builds nothing read, no per-write projector requests; the projector sweep releases every mark with nothing to project (search-index marks too while dormant) and only starts a pass for content writes; processing recovery skips dormant search-index documents; member-sync reclaims use their partial index
  • Removes unreachable code (knowledge/table VFS mutations, member-connectors and GitHub installation routes/contracts/hooks, bulk document delete, and other unused helpers)
  • Follow-up (separate PR after this deploys): a migration scoping the projection mark and ACL-mirroring triggers to search-index knowledge bases

Type of Change

  • Improvement

Testing

Type-check, lint, check:audits, unit suites, and the knowledge/sim-search integration suites against real Postgres. New integration coverage: workspace document-decided access (incl. a gated GitHub-installation source and a positive principal), member-sync reclaim scoping, processing recovery while dormant, projection work detection and mark release in both gate states.

Checklist

  • Code follows project style guidelines
  • Self-reviewed my changes
  • Tests added/updated and passing
  • No new warnings introduced
  • I confirm that I have read and agree to the terms outlined in the Contributor License Agreement (CLA)