fix(mcp): refresh remote MCP auth headers on SIGHUP reload (REPL-29064)

Merged
PI
4mo ago
+230-21

Abstract

Fixes the root cause of REPL-29064: after a SIGHUP env rotation, chat keeps working but every bedrock MCP tool call 401s forever, because live MCP connections keep the Authorization header they captured at connect time. This PR makes the SIGHUP harness reload swap freshly-resolved config headers into the live remote transports in place — the next tool call authenticates with the rotated JWT, with no reconnect, no tools/list refetch, and no disruption to in-flight calls.

Why this shape (auth-only refresh, not reconnect)

StreamableHTTPClientTransport / SSEClientTransport call _commonHeaders() on every request and re-read requestInit.headers fresh each time (SDK 1.29, normalizeHeaders(this._requestInit?.headers)). So rotating credentials requires zero connection churn: each remote server now owns one mutable headers object, shared with its transport, and MCP.refreshHeaders rewrites its contents from the freshly-invalidated config. The mcp-session-id is preserved; an in-flight tool call is untouched (the swap is synchronous on the event loop).

The alternative — reconnecting each server on reload (createAndStore) — would also refresh tool lists and structural config, but closes the previous client, which can kill an in-flight tool call, and pays the bedrock cold-start tail on every rotation. Since tool schemas don't change on env rotation, that cost buys nothing here. Structural MCP config changes (URLs, added/removed servers, stdio env) still require a restart, as before.

What changed

  • mcp/index.ts: remote connects now pass an owned mutable copy of mcp.headers to both transport candidates and register it in per-instance state (init path, storeClient, cleaned up in closeClient/onclose/finalizer). New refreshHeaders() service method re-resolves config per connected server and swaps header contents in place. It deliberately no-ops when the instance never touched MCP (InstanceState.has) so SIGHUP never triggers eager MCP connects.
  • harness-reload.ts: runs mcp.refreshHeaders() per instance after the cache invalidations, so it reads the already-fresh config (incl. rotated credentials from OPENCODE_CONFIG_CONTENT). Doc comment updated — MCP connections still survive reload.
  • Servers added dynamically via add() keep the headers they were added with (no fresh source to re-resolve from).

Tests

  • test/mcp/refresh-headers.test.ts (new): a connected remote server's live transport picks up rotated opencode.json headers after invalidate+refresh, same client/transport object (no reconnect, asserted via client construction count); refresh on an instance that never used MCP creates no transports/clients.
  • harness-reload.test.ts: reload invokes refreshHeaders once per loaded instance.
  • bun test test/mcp/ test/server/... test/session/prompt...: 71 MCP-suite + 55 session tests pass; tsgo --noEmit clean.

Context

  • Root-cause dossier: chat survives rotation because provider caches are invalidated on SIGHUP, but the old comment here said "MCP is intentionally NOT invalidated… MCP config changes still require a restart" — correct for connections, fatal for rotating credentials. Production symptom: 27h-uptime opencode, token expired mid-day, 100% bedrock tool 401s across sessions until process restart.
  • Coordinator-side mitigation that also works today: POST /refresh-integrations-mcp (full reconnect); this PR makes that unnecessary for rotation.

🤖 Generated with Claude Code

Summary by cubic

Fixes REPL-29064 by refreshing remote MCP Authorization headers on SIGHUP so rotated tokens are used immediately without reconnects or tool list refetch. Prevents persistent 401s from bedrock MCP tools after env rotation.

  • Bug Fixes
    • Each remote server now keeps a mutable headers object shared with its transports; the SDK re-reads headers on every request.
    • Added MCP.refreshHeaders() to re-resolve config and swap header values in place; no reconnect and in-flight calls are unaffected.
    • harness-reload calls mcp.refreshHeaders() after env/config/provider invalidation, applying new creds per instance while keeping connections open.
    • Skips instances that never used MCP; structural MCP config changes (URLs, added/removed servers, stdio env) still require a restart.