feat(docs,api,marketing): docs MCP server, /mcp alias, agent_auth identity types, reinstated OpenAPI spec

Merged
SA
4mo ago
+828-21

Links

  • Rationale + growth/debt ranking: plans/done/20260722-agent-readiness.md
  • Follow-up to #5910 (round 1, merged); orank rescan came back 70/100, this round targets >85

Summary

  • Docs MCP server: docs.superset.sh/mcp now speaks MCP — docs_search + docs_read tools over the live fumadocs source (stateless Streamable HTTP, no auth, read-only public content). The docs page moves to /mcp-server; browser GETs get a 308 so old links keep working. This turns the scanner's "docs MCP detected but unhealthy" into a real learn-surface over the same protocol agents act with.
  • Reinstated /openapi.json (revert of the drop merged in #5910): the rescan added a 5-pt "scoped permissions" check on top of the spec/schema/function-calling checks — ~16 pts total read securitySchemes and operations from an API spec. Same deliberately narrow scope: MCP transport, RFC-shaped OAuth endpoints, discovery docs; the 27 tools remain self-described via tools/list and the live server card, never hand-listed.
  • agent_auth completed with identity_types_supported: ["anonymous"] + per-type block. "Anonymous RFC 7591 registration, user claims via browser consent (PKCE)" is an accurate description of our flow, and the rescan confirmed the block is being read (register_uri was detected).
  • /mcp alias on api.superset.sh (re-export of the v2 handler, own maxDuration) and a superset.sh/mcp 307 redirect (preserves POST for MCP clients).
  • Smaller: /api/llms.txt scoped index + robots carve-out; .md twins and Accept negotiation extended to changelog pages; applicationSubCategory: "Developer Tools" in SoftwareApplication JSON-LD (nudges category classifiers away from "AI & ML").

Manual QA Checklist

  • Docs MCP: initialize, tools/list, tools/call (docs_search + docs_read) verified against local dev server; stateless per-request transport (same pattern as the api v2 route)
  • GET /mcp without Accept: text/event-stream → 308 to /mcp-server; /mcp-server page renders; nav updated (meta.json)
  • api /mcp alias returns 401 + WWW-Authenticate: Bearer resource_metadata=... unauthenticated
  • AS metadata carries the full agent_auth block (skill, register_uri, revocation_uri, identity_types_supported, anonymous.credential_types_supported)
  • /api/llms.txt serves; robots.txt allows it while still disallowing /api/
  • /changelog/<slug>.md returns text/markdown; drafts excluded
  • Production builds pass (docs verified this round; marketing + api verified in #5910 and unchanged structurally)

Testing

  • bun run typecheck (marketing, api, docs — re-run on this branch after cherry-picking onto latest main)
  • bun run lint (exits 0)
  • bun run build (docs)

Design Decisions

  • OpenAPI reversal: we dropped the spec in #5910 as content agents can't use. The rescan changed the math — the new scoped-permissions check plus three others only read from an API spec (~16 pts). Keeping it protocol-stable-only caps the maintenance cost: it needs updating only if the OAuth or MCP transport surface changes.
  • Docs page rename over method-splitting hacks: a route.ts owns /mcp entirely, so the page had to move; a 308 for browser GETs preserves every existing link.

Known Limitations

  • Docs MCP search is a simple term-frequency scorer over titles/descriptions/body — good enough for agent retrieval; not a search product.
  • Remaining scanner points beyond ~90 are external actions (Wikipedia/Wikidata, skills.sh npx skills add, ChatGPT directory, PyPI SDK) or a real product feature (MCP Apps ui://, ~6 pts) — documented in the plan doc.

Risks / Rollout

  • Risk: docs.superset.sh/mcp URL changes meaning (page → MCP endpoint). Mitigated by the 308 redirect for browsers; all internal links updated to /mcp-server.
  • Rollback: revert the PR — additive routes, one page rename, no data changes.

Summary by cubic

Adds a public docs MCP server, restores /openapi.json, and completes agent_auth metadata. Also adds an /mcp alias and small discoverability updates.

  • New Features

    • Docs MCP server at https://docs.superset.sh/mcp with docs_search and docs_read (Streamable HTTP, no auth). The docs page moved to /mcp-server; GET /mcp redirects (308) for browsers.
    • Reinstated https://api.superset.sh/openapi.json (OpenAPI 3.1) covering MCP transport and OAuth/discovery endpoints; tools remain self-described via tools/list.
    • agent_auth now includes identity_types_supported: ["anonymous"] with a per-type block describing anonymous DCR + PKCE consent.
    • /mcp alias on api.superset.sh to the v2 MCP handler; https://superset.sh/mcp 307-redirects to it (preserves POST).
    • GET /api/llms.txt added and allowed in robots.txt; .md twins + Accept negotiation extended to changelog pages.
    • Link headers and API catalog now point service-desc to the OpenAPI spec; docs and server-card URLs updated to /mcp-server.
    • applicationSubCategory: "Developer Tools" added to SoftwareApplication JSON‑LD.
  • Dependencies

    • Added @modelcontextprotocol/sdk to docs.

Summary by CodeRabbit

  • New Features

    • Added a public OpenAPI 3.1 specification covering API, MCP, OAuth, and discovery endpoints.
    • Added a documentation MCP server with searchable and readable documentation tools.
    • Added /api/llms.txt for machine-readable developer resources.
    • Added MCP aliases, documentation links, and improved service discovery metadata.
    • Added changelog support for Markdown content and content negotiation.
  • Documentation

    • Added comprehensive MCP Server setup, authentication, tools, and client configuration guidance.
    • Documented anonymous identity support and API key authentication options.
  • Bug Fixes

    • Updated API error hints and documentation links to reference current resources.