fix(chat): preserve tool policy across forced and final steps

Merged
WE
10h ago
+532-105

Description

Keep tool definitions on final-answer requests while disabling tool execution. Removing definitions changes the prompt prefix and prevents cache reuse. Enforce the step limit before tool execution, including when a provider ignores the request's tool choice.

Force the selected operation by name when supported. Keep existing Claude reasoning behavior and Qwen/GLM compatibility. Forced tools with multiple operations retain the full operation group. Recover plain-JSON tool calls for named choices through the shared response parser. Update the shared mock provider and integration expectations to support named choices.

Bedrock final-answer requests omit tool definitions because Converse cannot honor NONE. The adapter preserves prior tool exchanges as text. Bare JSON recovery uses the selected tool; explicitly named mismatches remain subject to execution checks.

This is the first layer of the chat-history stack. Tool-result retention and stable prompt placement follow separately.

Controlled Claude Haiku 4.5 tests: keeping tool definitions reduced total cost by 43% across four calls and by 92% on the final call. These are synthetic provider measurements, not production averages. Raw usage, prices and reproduction scripts.

OpenAI measurements: keeping tool schemas reduced GPT-4.1 mini cost by 27% across three calls (67% on the final call). Raw usage, reproduction scripts and the GPT-6 Luna explicit-cache experiment are included.

How Has This Been Tested?

  • 77 focused unit tests passed on this layer alone. Another 103 recovery/runtime tests passed; named complete-response and streaming regressions fail before the recovery fix.
  • 118 external dependency tests passed with the result-retention layer, including the real HTTP mock-provider contract.
  • Commit checks passed, including types, lint and feature-map integrity.

Docs

  • Customer-facing change: I updated documentation/ (text and screenshots).
  • No docs change needed (internal, tests, CI, or multi-tenant cloud only).

Updated the tool-framework contract. No UI or configuration changes.

Additional Options

  • [Optional] Please cherry-pick this PR to the latest release version.

Summary by CodeRabbit

  • Bug Fixes
    • Tool selection now respects explicitly named tools, preventing other available tools from running during that step.
    • Tool-choice handling is more consistent across model types, including requests involving multiple operations and final-step limits. At the step limit, tools cannot run; supported providers may still receive tool definitions.
    • Text-based tool-call recovery now works with named tool choices, helping preserve the selected operation when interpreting model responses.
  • Documentation
    • Clarified that a forced ID can select a group of OpenAPI operations, with the model choosing which operation to run.