fix(solutions): forward explicit solution-YAML BM25/RRF values by presence (#1076)

AD
4d ago
+944-68

What

A solution YAML bm25_k1: 0 / rrf_k: 0 was still silently replaced by 1.2 / 60: RAGConfig.bm25_k1 / bm25_b / rrf_k were plain proto3 scalars, so expand_rag could only test the value (> 0) and an explicit 0 never reached the retrieve operator's range check. bm25_b: 0 (a legal value: no length normalization) was dropped the same way.

  • idl/solutions.proto: the three RAGConfig fields become proto3 optional (wire-compatible), so presence survives YAML → proto.
  • solution_converter.cpp: forwards each field by presence, so the retrieve operator sees the key and its existing guard rejects bm25_k1 <= 0 / rrf_k <= 0, while bm25_b = 0 is honored instead of dropped.
  • config_loader.cpp: parses the three keys strictly (whole scalar, finite, classic locale). to_float() used to turn garbage like bm25_b: "abc" into 0, which presence would now forward as a legal b = 0; it is a load-time RAC_ERROR_INVALID_FORMAT naming the key instead.

Tests

core/tests/test_solution_runner.cpp (rag_yaml_explicit_zero_tuning_reaches_the_retrieve_operator): a YAML with explicit zeros carries has_* presence and emits bm25_k1/bm25_b/rrf_k = "0" on the retrieve operator (where the existing retrieve_rejects_malformed_params cases reject the k1/rrf zeros); an unset key forwards nothing; a garbage value fails to load with a rag.bm25_b detail.

Stacking

Stacked on #1065 (whose operator guard does the rejecting), which sits on #1048 → #1017, so the diff includes those commits until they merge. idl/SCHEMA_LOCK is regenerated for the solutions.proto change.

Closes #1076

Summary by cubic

Solution YAML and per-query RAG retrieval now forward explicit BM25/RRF tuning and vector-store selection by presence, so bm25_k1: 0, bm25_b: 0, and rrf_k: 0 no longer collapse into defaults.

  • RAGConfig and RAGRetrievalOptions make bm25_k1, bm25_b, rrf_k, and vector_store proto3 optional fields, wire-compatible, so explicit zeros survive YAML → proto.
  • The solutions retrieve operator, session config, and query overrides validate these values: bm25_k1/rrf_k must be positive and finite, bm25_b must lie in [0, 1], and any non-USEARCH vector store is rejected on-device.
  • Solution YAML parsing is now strict: trailing characters, non-finite floats, and out-of-range values fail to load with a rag.<key> error instead of silently substituting the default.
  • BM25 scoring and RRF fusion take k1, b, and rrf_k at call time; RRF scores accumulate in double with deterministic tie-breaking.
  • Kotlin, Python, Swift, and web SDKs expose bm25K1, bm25B, rrfK, and vectorStore on RAG retrieval options.

Bug Fixes

  • Explicit bm25_k1 = 0 and rrf_k = 0 are rejected with an error rather than replaced by the default; bm25_b = 0 (no length normalization) is honored.
  • Malformed values like bm25_b: "abc" fail at load time instead of silently becoming a legal 0.

Summary by CodeRabbit

  • New Features
    • Added per-query controls for BM25 ranking, reciprocal-rank fusion, and vector-store selection across Kotlin, Python, Swift, and web SDKs.
    • Added USearch vector-store selection and session-level configuration for RAG ranking settings. Unset options continue to use session defaults.
  • Bug Fixes
    • Improved validation of retrieval settings, with clearer errors for malformed, out-of-range, or unsupported values.
    • Made floating-point RAG settings parse consistently across locales.