feat(workspace): attach the bound workspace's engine as a derived MCP overlay

Merged
RA
2mo ago
+2791-57

Issue for this PR

Closes #1153

Type of change

  • Bug fix
  • New feature
  • Refactor / code improvement
  • Documentation

What does this PR do?

Lets a terminal session use the integration tools of the workspace its project is bound to, served by the local engine (@altimateai/datamate 0.7.0 or newer, whose --datamate pin is locked) instead of falling through to the hosted endpoint, which serves a different tool set. Behind the workspace pilot flag; terminal only.

The mechanism is deliberately small. In the config loader, after external MCP discovery, a bound directory gets its datamate MCP entry derived: datamate start-stdio --datamate <id>, if an engine on PATH clears the version floor. It is never written to any file, and it has the last word over IDE-written, hosted and stale entries under that key. MCP bootstrap then starts it like any other stdio server and awaits it before the first tool list, so the tools are present on turn one with no extra wait code.

Each turn boundary (before tools are resolved):

  • re-reads the binding; on a re-link it reloads the overlay and replaces the engine, on an unlink it removes it;
  • if the engine's handshake failed, re-adds it once per process, then reports;
  • settles the session's outcome (disabled, unbound, attached, engine-missing, engine-too-old, connect-failed) — a pure read for other modules;
  • announces once per verdict: the declared-vs-delivered inventory when attached, or a refusal naming the exact install command. Headless run prints one stderr line instead (stdout stays JSON-clean).

In workspace mode the in-process writers of the datamate key — the IDE reload endpoint, the HTTP MCP add route and datamate_manager add — refuse it with "managed by workspace". Standalone datamate- entries are untouched. The serve command (the extension host) is excluded so the extension's own engine and bridge are never overridden.

Also: MCP.remove now forgets the key's runtime config (previously it kept reporting "disabled" and connect re-spawned the removed entry); the integrations listing hides extension-type entries with a count; API requests carry a 15 s abort that stays armed until the body is read.

What this does not claim. The engine serving a turn is the one launched for the binding read at that turn's start. A change made by another process (an editor, a link from another shell) is observed at the next turn boundary, not instantly; a re-link landing mid-turn takes effect next turn. An explicit enabled:false on the key is overridden in workspace mode (opt out by unlinking or turning the flag off). A crash-free process holds one engine per key; a second altimate-code process on the same project starts its own, as with any stdio MCP entry. Switching accounts via browser login mid-turn keeps that turn's already-catalogued tools on the previous account until the next turn. About 4 % of engine starts stall at the handshake (a separate, pre-existing engine issue); this retries once and then says so.

How did you verify your code works?

Unit: test/altimate/workspace — 58 tests over the overlay (what the config loader gets under flag/serve/unbound/bound × engine present/missing/old/broken; every turn-boundary path: attach + inventory, refusal + dedupe, headless line, retry-then-fail, retry-then-succeed, re-link, unlink, engine installed after a refusal, probe TTL, bounded session memo, hook never throws) and the pure vocabulary (SemVer floor incl. pre-release and unreadable cores, pin parser in both IDE spellings, the outcome tables over the whole union). The MCP lifecycle suite gains the remove case. Affected suites: 243 pass / 0 fail; config/prompt/tools suites 448 pass / 0 fail; typecheck clean.

End-to-end on a real bound workspace: pending — rows to run on the maintainer's rig are listed in the review-log comment below and will be recorded there before this leaves draft.

Screenshots / recordings

Not a UI change; toasts are TUI-only and will be captured as pane output with the E2E rows.

Checklist

  • I have tested my changes locally
  • I have not included unrelated changes in this PR

Summary by CodeRabbit

  • New Features
    • Workspace-linked DataMate engines now connect automatically, refresh when bindings change, and pin tools for each session turn.
    • Added status notifications for missing, outdated, unavailable, or unlinked engines.
  • Bug Fixes
    • Prevented manual MCP changes from overriding workspace-managed DataMate connections.
    • Removed MCP servers are now fully removed and no longer reappear as disabled.
    • API requests now time out safely after 15 seconds.
  • Improvements
    • Headless and server sessions report engine status more reliably.
    • Workspace-managed connection errors provide clearer feedback.
    • Extension-managed integrations are hidden from integration listings.
    • Workspace connections now respect credential scope changes.