feat(codemode): let a plugin rank execute's catalog and index tool families

Merged
YU
1mo ago
+372-45

Lets a plugin choose which tools execute's catalog shows, ranked by real use, and adds a one-line index per tool category so the rest stay discoverable. With no plugin options the catalog output is byte-identical to today.

Today the catalog fills its ~2,000-token budget cheapest line first, so on a typical Replo project it shows 23 of 118 Bedrock tools that together receive about 6% of real calls, while the most-called tools (product reads, generation status, image generation) never appear. With the andytown harness passing its ranked list, the same budget shows the tools that receive about 55% of calls.

  • packages/codemode: prepare accepts optional pinned (ranked paths), pinnedLimit and categories ({ namespace, name, match, description }, a name-pattern → one-line description). Pinned tools are placed first in rank order within the existing budget; absent tools are skipped; a category line prints only while that category has unshown tools.
  • tool.definition hook for execute may set output.discovery; invalid options log a warning and fall back to the default catalog, so a bad harness deploy cannot remove execute.

Naming follows prior art: OpenAI tool search groups deferred tools as namespaces with a name and description, Anthropic's tool-search docs call these tool categories, and MCP's groups proposal (SEP-1300) uses name/description. namespace already means an MCP server in Code Mode, so the sub-groups are categories; pinned matches the "always loaded" tools those docs describe.

Consumer: replohq/andytown yuxin/agent-tools-overhaul (REPL-32929), which will run on an opt-in prerelease of this branch for its benchmark before any stable pin bump.

Testing Done

  • Code-mode tests: 271 passed; tool registry + code-mode: 73 passed (2 new). After the rename: codemode.test.ts 51 passed, registry.test.ts 19 passed; the pre-push hook's 30 typecheck tasks pass.
  • Typecheck clean in the three changed packages.
  • Rendered the catalog offline for three real projects at v1.18.32-7: default output byte-identical without options.

Summary by cubic

Lets a plugin rank which MCP tools execute inlines in its catalog and index the rest by category.

Today the catalog fills its ~2,000-token budget cheapest line first, so a typical project shows 23 of 118 Bedrock tools that receive only ~6% of real calls while the most-called tools never appear. With a ranked list from the harness, the same budget shows tools that receive ~55% of calls. Without plugin options the output is byte-identical to today.

  • codemode prepare accepts optional pinned (ranked tool paths), pinnedLimit, and categories (prefix → one-line summary); pinned tools are inlined in rank order within the existing budget, absent tools are skipped, and a category line prints only while that category has unshown tools.
  • The tool.definition hook for execute may set output.discovery; invalid options log a warning and fall back to the default catalog so a bad harness deploy cannot remove execute.
  • The andytown harness consumes this on an opt-in prerelease of this branch before any stable pin bump.

Devin Review