feat: per-turn tool retrieval (trim tool-definition context flood)

Merged
AN
4mo ago
+203-0

What does this PR do?

Adds a flag-gated (ALTIMATE_TOOL_RETRIEVAL, default OFF) per-turn tool subset: an always-on core set plus a lexically-ranked top-k of the remaining tools, never dropping a tool already referenced mid-trajectory. This trims the ~78-tool definition flood that hurts tool selection and inflates input tokens. v1 is lexical, dependency-free, and deterministic. Wired into session/llm.ts (marker-wrapped); a no-op when the flag is unset.

Previously this PR also carried constrained decoding and a critic gate. Both were split out so each lands on its own merits:

  • critic gate → wired, tested, reviewed in #863 (closes #862).
  • constrained decoding → local-providers-only and not yet validated; moved to feat/constrained-decoding, tracked in #868.

Validation (measured)

Tool retrieval — A/B, 8 ADE-bench dbt tasks, deepseek-v4-flash, value-graded (output checked for correctness, not just "builds"):

ArmResolvedInput tokensCost
ALTIMATE_TOOL_RETRIEVAL=14/8238k$0.024
baseline (all ~78 tools)4/8474k$0.048

→ identical resolve-rate (0 tasks differ), −50% input tokens, −49% cost.

Type of change

  • New feature (non-breaking change which adds functionality)

Issue for this PR

Closes #857

How did you verify your code works?

  • 5 unit tests (test/tool/retrieval.test.ts) — selection logic incl. always-on core + referenced-tool retention; all green.
  • Tool-retrieval A/B (above) — value-graded on real dbt tasks: −50% input tokens / −49% cost at identical resolve-rate.
  • tsgo typecheck clean; altimate_change markers in session/llm.ts balanced (7/7); default-off so the non-flagged path is unchanged.

Checklist

  • My code follows the style guidelines of this project
  • I have performed a self-review of my code
  • I have added tests that prove my feature works
  • New and existing unit tests pass locally with my changes

Summary by CodeRabbit

  • New Features

    • LLM now uses an optional tool-retrieval mechanism to trim the active toolset per query (disabled by default; enable via env).
    • Retrieval always preserves core tools and any in-flight referenced tools while selecting additional relevant tools.

  • Documentation

    • Added guide describing tool retrieval behavior and configuration.

  • Tests

    • Added tests covering selection behavior, preserved tools, edge cases, and enablement toggling.