Sync MCP server tool lists through the MCP SDK and cache them on the entity

Merged
AB
14d ago
+2489-1335

Second step of exposing external MCP servers to the AI chat, after twentyhq/twenty#28597. Twenty now talks to a registered MCP server and caches its tool list, so later turns never fetch tools/list live.

What

  • mcpServer.tools (jsonb), toolsSyncedAt, toolsSyncError, with a 2.53 fast instance command and @WasIntroducedInUpgrade on the new columns.
  • McpServerClientService.listTools over @modelcontextprotocol/sdk Streamable HTTP. The transport uses the SSRF-guarded axios client from SecureHttpClientService (new outbound source mcp-server), one 20s deadline across the handshake and every tools/list page, at most 10 pages, stateless connect per call. A linked connection provider authenticates the server: the account is picked by the provider scope (the workspace-shared account, or the calling user's own for USER scope; archived or auth-failed accounts skipped, MCP_SERVER_NOT_CONNECTED when none), tokens come from ConnectedAccountRefreshTokensService.resolveTokens (refreshed when stale) and are decrypted per workspace.
  • McpServerToolSyncService: fetches tools, validates each one with a zod schema (mcpServerToolSchema: name must match ^[a-zA-Z0-9_-]{1,64}$, description and title stripped of control characters and capped at 1024 chars, only the standard annotation hints kept, nested input schema nodes kept opaque so the jsonb column stays typeable) and writes the result through the workspace-scoped repository, then invalidates flatMcpServerMaps. It never throws: an unreachable server leaves the settings saved and the reason in toolsSyncError.
  • Sync runs on create, on update when url or connectionProviderId changes, and on demand through the new syncMcpServerTools(id) mutation. The DTO exposes tools, toolsSyncedAt, toolsSyncError.

Why always the axios-backed fetch

createSsrfSafeFetch() hands back the global fetch when every host is allowed. The integration harness replaces that global with node-fetch for msw, whose non-WHATWG response body stalls the SDK's SSE reader, and in general the SDK only needs a WHATWG-shaped response. Going through getHttpClient() keeps the allowlist, timeouts and request logging on one path in every environment. Verified against https://docs.twenty.com/mcp (SSE responses) with the same fetch.

Why the tool cache lives on the entity and is written directly

The cached tools are derived from the remote server, never authored by a user or an app manifest, so they are not part of the syncable definition: tools, toolsSyncedAt and toolsSyncError are toCompare: false in the properties configuration, a manifest sync neither sets nor diffs them, and the manifest converter only seeds them empty. Routing every sync through the workspace-migration builder would turn each refresh into a migration action and a cache rebuild for a value the builder cannot validate. The direct workspace-scoped repository write followed by a flatMcpServerMaps invalidation keeps the flat cache the single read path the tool provider needs on every chat turn, which is the reason the list is cached at all. Keeping them on the entity rather than in a side table means one read, one cache and one GraphQL type; if the per-workspace cache ever grows too large, the column can move to a keyed cache without a schema change for consumers.

Follow-ups

  • McpToolProvider exposing cached tools in chat and workflows, together with McpServerClientService.callTool (next PR).
  • Hand-added servers with a plain API key need an apiKey/pat connection provider type; today a server is either public or OAuth-backed through a connected account.

Test

  • test/integration/metadata/suites/developers/mcp-server-tool-sync.integration-spec.ts runs an in-process MCP server (mcp-server-fake-server.util.ts) with SSRF safe mode on and 127.0.0.1 allowlisted: tools cached on create with the invalid-name tool dropped, the linked provider's shared account token sent as a Bearer header, unreachable server records toolsSyncError, re-sync on url change and on demand keeps the last good list.
  • sanitize-mcp-server-tool.util.spec.ts for the sanitizer.
  • Existing mcp-servers and endpoint-permission specs pass; tsgo --noEmit on server and front, lint clean.