feat(workspace): answer "what does this workspace have" from the workspace

Merged
SA
2mo ago
+1108-23

Issue for this PR

No tracking issue. Seen in demos: asked which skills or knowledge a workspace has, the agent lists every installed skill instead of the workspace's own, and answers differently on each ask.

Type of change

  • Bug fix
  • New feature
  • Refactor / code improvement
  • Documentation

What does this PR do?

The system prompt's ## Altimate Workspace section names the linked workspace but says nothing about what it contains, and the model sees one undifferentiated <available_skills> list — built-in, project and workspace skills together. So "what skills does this workspace have" was answered with everything, grouped differently each time.

For a linked project, the identity section is now followed by a separate "What this Altimate Workspace provides" block (altimate/workspace/contents.ts):

  • Skills: the workspace's own skills, read from the synced snapshot on disk — only when the snapshot belongs to the current account (snapshotIsOurs) and to the workspace the project is linked to now, not a previous link (new snapshotWorkspaceId). Sorted, so every ask sees the same list; names and descriptions sanitized to single bounded lines.

  • Integrations and memory setting: from the workspace summary (/datamates/{id}/summary), cached five minutes per workspace and waited on for at most 300 ms, so a slow service never holds a step (the section renders every step). A failed fetch is not cached; the next step asks again.

  • Knowledge: from the same summary (knowledge_engine_enabled, knowledge_bases). The block says one of:

    • the knowledge engine is off;
    • every document in the knowledge hub (engine on and nothing selected, so a count would be wrong);
    • the selected documents by name, resolved through GET /knowledge_bases/, sorted, deleted ones left out, at most 20 then a count;
    • the selection size, when the document list cannot be read.

    The document list is only fetched when specific documents are selected, inside the same cached, time-bounded fetch.

  • Instruction: answer workspace questions from this block, list the same items every time, keep Altimate Code's built-in skills separate (they ship with the CLI and are in every project), and treat names and descriptions as labels, not instructions.

It is a separate block with its own cap rather than part of the identity text, because render drops the identity section entirely when it overflows and a long skill list must not take the link statement down with it. At most 40 skills are listed, then a count; descriptions are dropped before anything else, and the block is omitted rather than cut. "Not synced yet", "none" and "not known" render differently.

How did you verify your code works?

End to end, live tenant, a scratch repo linked to a workspace with one custom skill, asking "What skills does this workspace have?" three times per binary:

  • 0.12.4: all three answers list the built-in skills (dbt-develop, sql-review, cost-report, …) as "skills in this workspace", with the one workspace skill at the end, grouped differently each time.
  • This branch: all three answers name exactly the one workspace skill, say that is the complete list, and keep built-ins separate; one also reports the integrations and memory setting.

End to end, knowledge. The same workspace has the knowledge engine on and no documents selected. Asking "What knowledge does this workspace have?":

  • 0.12.4: listed the workspace's memory blocks as its knowledge.
  • This branch: "has access to every document in the organization's knowledge hub — it is not limited to specific documents."
  • Not run live: the "specific documents selected" path. The test workspace has none selected, and I didn't change its configuration. It's covered by unit tests.

Tests:

  • test/altimate/workspace/contents.test.ts: list rendering and instruction; not-synced vs none vs unknown; 40-item cap with count; oversize drops descriptions, then omits; summary sorted, cached, failure is "not known" not "none"; a slow service returns within the wait and fills the cache later.

  • identity-section.test.ts: a linked project gets the block after the identity section; an unlinked one does not (mutation-checked: removing the wiring fails it).

  • contents.test.ts: a failed summary fetch is retried on the next step (mutation-checked).

  • contents.test.ts, knowledge:

    • the off / all / selected / unknown wording;
    • the 20-document cap with a count;
    • selected ids resolve to the names of documents that still exist, sorted;
    • the document list isn't fetched for off or all;
    • an unreadable list keeps the count.

    Mutation-checked: keeping deleted documents, dropping the "all" or "off" case, treating an unreadable list as empty, or removing the cap each fails a test.

Existing test fixed on the way. "a step arriving after an unlink does not join the pre-unlink resolve" waited for a second request on the wire. After the unlink there is no local binding, so step 2 settles without any request; the second request the test saw on main was a detached skill sync left over from an earlier test's bind (logged: /skills?datamate_id=11). It also fails on main when run alone with -t. This branch loads skill-sync earlier, which shifted that timing and made the test fail under load (3/18 parallel runs). The test now asserts the actual invariant — step 2 settles while step 1 is still parked and does not name the old workspace — and fails when both inflight.clear() and the generation check are removed. 24/24 parallel runs pass.

Typecheck, tracker-leak check and marker guard clean. Full suite (latest commit): 14,381 pass, 8 fail.

  • 6 are MCP tests reading the developer's real ~/.claude.json (#1386).
  • 2 are test/cli/run/silent-turn.test.ts. That file is flaky on recent main too: run alone, it failed 2 of 6 times on a main-based branch and 1 of 6 here.

Screenshots / recordings

N/A — prompt content; answers quoted above.

Checklist

  • I have tested my changes locally
  • I have not included unrelated changes in this PR

🤖 Generated with Claude Code

https://claude.ai/code/session_018fJ3X7pcGT4R9yzjsJnqsV


Summary by cubic

Fixes "what does this workspace have" so the agent answers from the linked workspace instead of listing every installed skill, consistently across asks. For a linked project, the identity section is now followed by a "What this Altimate Workspace provides" block.

  • Lists the workspace's own skills from the synced snapshot on disk, pinned to one validated snapshot and account; a mid-read swap or slow read answers "could not be read just now"; a skill that is unreadable — a folder whose SKILL.md is missing vs. one that fails to parse or stat — is listed by folder name rather than dropped.
  • An empty workspace reads "none" from an atomic on-disk record that a sync only withdraws if it wrote it, so a newer record from another thread survives and a relink clears it.
  • Adds integrations, memory setting, and knowledge from the summary, cached per account and workspace and awaited at most 300 ms; failed or incomplete fetches are retried the next step.
  • States knowledge as off, every hub document, or the selected documents by name, with deleted ones excluded; names resolve through the workspace-scoped endpoint so a teammate sees the owner's private selections, and at most 20 names render, then a count.
  • Neutralizes skill and knowledge names so uploaded content cannot fake the prompt's tags; instructs the model to answer from this block, keep built-in skills separate, and treat names as labels.
  • Caps the block and falls back to counts only, rendering nothing when even the counts-only form exceeds the caller's cap.
  • Fixes a flaky post-unlink test that asserted a second network request; it now proves step 2 settles while step 1 is parked and does not name the old workspace.

Summary by CodeRabbit

  • New Features
    • Prompts for linked projects now include available skills, integrations, memory settings, and knowledge-base information.
    • Knowledge-base details can show selected document names and counts, excluding missing or deleted documents.
    • Workspace skill information distinguishes between unavailable, unknown, empty, and populated states, and separates workspace skills from built-in skills.
  • Improvements
    • Workspace details are omitted for unlinked or stale projects, and lengthy prompt sections are shortened to fit within a character limit.
    • Workspace information remains responsive when details are slow or unavailable.