feat(chat): replace forced Actions with tool preferences

Closed
WE
12h ago
+1861-827

Description

Replace forced Actions with an advisory tool preference. Keep the existing persistent selection in web and mobile. Users can select an Action, but the model can use another available tool or answer without tools. This removes the conflict between forced execution and the stable tool contract in the preceding PRs.

The preference appears in the trailing request reminder. It survives retries and restoration within the turn, but does not enter recorded history. Web and mobile keep the selection across messages and send it with each request until the user changes it.

API clients must replace forced_tool_id with preferred_tool_id. Non-null legacy forcing and preferences combined with Deep Research return validation errors. Slack and eval callers use the same advisory contract; tool-use assertions remain explicit in evals.

How Has This Been Tested?

  • Three independent backend review rounds, plus parent reviews of ownership, duplication, client races, legacy callers, and hot paths. Valid findings were fixed before publication.

  • Seven integration tests through the running frontend and API, with isolated PostgreSQL, Redis, search, model server, and Celery workers. Scripted LLM responses verify optional use, follow-up isolation, availability checks, allowlists, parallel calls, and XML tool-call recovery.

  • Final backend runs: 1,337 runtime/chat/API tests and 352 eval/Slack/benchmark tests passed. The post-rebase chat-agent run passed 50 tests.

  • Persistent-selection follow-up: 36 web tests and 106 mobile tests passed, including repeated submission, chat handoff, and follow-up selection. Full type checks and lint passed.

  • Normal commit hooks passed. A live Anthropic test passed for evidence replay, compaction, and restoration; its follow-up read about 99% of input tokens from cache. Tool setup in this test is deterministic. Live preference compliance rates were not measured.

  • Additional document-set integration coverage is blocked locally by a missing enterprise license (HTTP 402). The MCP OAuth suite requires Nginx, which is not installed locally; its scripted AUTO expectation was corrected but the suite could not run.

Docs

  • Customer-facing change: I updated documentation/ (text and screenshots).

Updated the chat API and image-generation guidance, eval documentation, and feature-map contracts. Inspected the local web Actions menu; the changed docs do not contain an affected screenshot.

Additional Options

  • [Optional] Please cherry-pick this PR to the latest release version.

Summary by cubic

Turns forced Actions into an advisory preference. Selecting an Action now only tells the model which tool it may use; the model can still pick another available tool or answer without tools, which removes the conflict forced execution had with the stable tool contract.

Behavior

  • The preference travels in a request-only trailing reminder and survives retries and restoration within the turn, but not recorded history. Web and mobile keep the selection across messages and send it until the user changes it. Preference prompts are centralized in chat_prompts.py and rendered by build_chat_prompt for the current step only.
  • API clients must replace forced_tool_id with preferred_tool_id; non-null legacy values are rejected with a validation error.
  • A preference combined with Deep Research returns a validation error, and an unavailable preferred tool is rejected instead of silently dropped.
  • Eval datasets reject legacy force_tools inputs; use preferred_tool with expected_tools assertions instead.
  • RunProgress.tool_configuration_digest is now required, so legacy checkpoints built before this change no longer restore.
  • The EnterpriseRAG benchmark script now fails a case whose answer does not show a completed internal search; the Slack handler reads the full response to detect that tool call.
  • When a project turns search off, the Search preference is omitted from that request while the UI selection remains active for later requests.

Summary by CodeRabbit

  • New Features
    • Select an Action to prefer in a chat. The assistant may use another Action or respond without using one; the preference carries across messages in that chat.
    • Web and mobile interfaces show the selected preference and let you change or clear it. Deep Research cannot be used at the same time as an Action preference.
  • Bug Fixes
    • Unavailable or unauthorized Actions now return a clear error rather than being silently ignored or treated as guaranteed to run.
    • Slack responses now handle completed search results more accurately, including when to show a no-indexed-data notice.