fix(rag): bound per-query top_k and score_threshold like the session config (#1102)

AD
4d ago
+897-53

What

The session config bounds top_k and score_threshold, but the per-query path (rag.query and rag.search) copied them straight through. A score_threshold outside 0...1 or NaN, or a negative top_k, was used as-is or silently fell back to the default, so retrieval mis-ranked with no error.

Both entry points now reject those values with RAC_ERROR_INVALID_ARGUMENT, through one validate_retrieval_bounds helper beside the BM25/RRF check.

One deliberate deviation from the issue: top_k == 0 stays valid, meaning "use the session default". The web SDK already sends topK: 0 for an unset value and documents it that way, so rejecting zero would break callers. Only negative top_k is an error.

Not done: the issue's third item (range annotations on the four proto fields). That is a schema change that also touches idl/VERSION and SCHEMA_LOCK, which are already being sequenced across the stack, so it is left for that step.

Verification

  • test_advanced_modality_proto_abi: all new checks pass (negative top_k, score_threshold 1.5, NaN score_threshold, and top_k 0 accepted). The only failures are the two existing Windows VLM model-path checks, which fail the same way without this change.
  • test_rag_fusion: 14 checks, 0 failures.

Stacking

Stacked on #1066 (fix/web-rag-knobs-cross-wasm-provider-1057), which sits on #1048 → #1017. The diff includes those commits until they merge.

Closes #1102

Summary by cubic

Per-query RAG retrieval (rag.query and rag.search) now bounds top_k and score_threshold like the session config, so a negative top_k or a score_threshold outside 0...1 (or NaN) is rejected with RAC_ERROR_INVALID_ARGUMENT instead of silently mis-ranking results. The PR also threads BM25, RRF, and vector-store knobs through the whole stack.

  • top_k == 0 stays the "use the session default" sentinel that every SDK already sends.
  • BM25 (bm25_k1, bm25_b), RRF (rrf_k), and vector_store are validated at the ABI boundary and wired into session config, per-query overrides, the retrieve backend, and all SDK bindings (Kotlin, Python, Swift, Web).
  • The retrieve operator parses floats with the classic locale and rejects malformed params; the converter formats floats the same way so they round-trip.
  • The Web cross-WASM RAG provider rejects the native-only knobs with an error instead of silently ignoring them.
  • VECTOR_STORE_USEARCH is whitelisted on-device; PGVECTOR returns NOT_SUPPORTED and unknown enum values return INVALID_ARGUMENT.
  • The VectorStore enum moved from solutions.proto to rag.proto and IDL_VERSION bumped to 1.7.0; proto range annotations on the new fields are left for a separate schema change.

Summary by CodeRabbit

  • New Features
    • RAG retrieval options now support per-query BM25 tuning, reciprocal-rank-fusion tuning, and USEARCH vector-store selection across Kotlin, Python, Swift, and Web SDKs.
    • Retrieval settings can be customized at the session level, with per-query overrides where supported. Omitted options continue to use session defaults.
    • Ranking results are more consistent when candidates have equal scores.
  • Bug Fixes
    • Invalid retrieval settings are rejected with errors instead of being silently accepted or misinterpreted.
    • Cross-WASM queries report an error when given native-only retrieval options.