feat(sdk,cli): SDK usage telemetry and client identity headers for the SDK and CLI

Merged
SA
2mo ago
+242-55

Summary

Instruments @superset_sh/sdk the way the CLI is instrumented so the "MCP & CLI Usage" dashboard can show SDK usage. The four SDK tiles on dashboard 1566665 read sdk_method_called, broken down by method and sdk_version; those names are unchanged.

  • Every public resource method reports one sdk_method_called event to analytics.captureEvent with source: "sdk" and properties: { method, sdk_version, sdk_lang, runtime, target, success, duration_ms }. method is the public name the caller typed (tasks.list, workspaces.create), never the tRPC path.
  • The report is issued from the APIPromise parse step inside the hand-written tRPC helpers, which now also owns the envelope unwrap: success: true only once the caller's data has resolved, false on transport, HTTP, or parse errors, and duration_ms runs to final settlement. It never sits in the caller's chain (no await, no throw, no retry, one 10 s timeout) and does not force a parse on callers that only want asResponse(), so those calls go unreported rather than double-reading the body. The capture goes through post, so it is not itself reported. A sync bug in telemetry is caught too.
  • SUPERSET_TELEMETRY=0 (or false) opts out. The CLI has no opt-out today, so there was no existing name to reuse.
  • Cloud requests send x-superset-client: sdk/<version>. ApiClientInfo.product and CLIENT_HEADER_PATTERN accept sdk. The only consumers of product are the Sentry tag in apps/api and the client_product property; neither switches on it exhaustively. No test fixtures reference the header.
  • Docs: a Telemetry note in apps/docs/.../sdk/reference.mdx and packages/sdk/api.md. The CLI reference says nothing about its telemetry, so there was nothing to mirror; the note is short.

Design notes

  • Stainless's CONTRIBUTING says the generator never touches src/lib/, so the telemetry module lives at packages/sdk/src/lib/telemetry.ts. Neither stainless-sdks/superset-typescript nor cadra-typescript wires analytics, and the generated client only exposes pre-request hooks (prepareOptions, prepareRequest), so observing the response promise in our own helpers is the smallest way to get success and duration_ms.
  • client.ts carries a "generated" header but has eight hand-edit commits (the tRPC and host helpers are ours, no Stainless config exists in the repo). The edits are confined to those helpers plus one header line in buildHeaders, next to User-Agent and the X-Stainless-* identity headers.
  • The helpers now take { method, procedure } instead of a bare procedure string, which touches all 32 resource call sites but keeps the label explicit rather than a mapping table that would drift.
  • VERSION in version.ts was still 0.0.1-alpha.7 while package.json and npm are at alpha.12, so every published SDK since alpha.8 has identified itself as alpha.7. scripts/build.ts now rewrites src/version.ts from package.json before bundling, so the published User-Agent, x-superset-client, and sdk_version always match. Publishing is manual (bun run build, then npm publish from dist/) with no CI path around it, so the stamp cannot be skipped; the build script header now documents the release steps and that version.ts is never hand-edited.
  • runtime reuses detect-platform.ts (X-Stainless-Runtime) and adds a process.versions.bun check, since Stainless reports Bun as node.
  • The CLI did not send x-superset-client at all (only desktop and mobile did), so CLI-caused API errors and CLI procedure usage were invisible to the same two consumers. It now sends cli/<version> from the headers() callback in packages/cli/src/lib/api-client.ts; the server already accepted cli. Dev builds send cli/0.0.0-dev, which matches the pattern.

Verification

  • bun run lint:fix, bun run typecheck, bunx sherif --fix: clean. bun test per package: packages/trpc 125 pass, packages/cli 143 pass. (A root-level bun test shows 258 unrelated failures in host-service, desktop, workspace-client, and pty-daemon that pass when run per package.)
  • No unit test: packages/sdk has no test setup and its tsconfig deliberately sets types: [], so a bun:test file fails its typecheck. Per the brief, no new test setup was introduced.
  • End to end against the local dev stack (API on the workspace's Neon branch, dev API key minted for satya@superset.sh, revoked afterwards): one client.tasks.list() produced GET /api/trpc/task.list 200 followed by POST /api/trpc/analytics.captureEvent 200, and the event arrived in PostHog project 264828 ("Local Development", confirmed as the owner of the NEXT_PUBLIC_POSTHOG_KEY in .env):
{
  "uuid": "01a06902-9f1a-71fb-be5b-fb5147b49811",
  "event": "sdk_method_called",
  "timestamp": "2026-09-03T20:42:54.859000Z",
  "distinct_id": "0da9ed59-9eb3-4f73-ac71-6f1df5a44816",
  "properties": {
    "source": "sdk",
    "method": "tasks.list",
    "sdk_version": "0.0.1-alpha.7",
    "sdk_lang": "js",
    "runtime": "bun",
    "target": "cloud",
    "success": true,
    "duration_ms": 2212,
    "active_organization_id": "b2c3d4e5-f6a7-4890-9bcd-ef1234567891",
    "plan": null
  }
}

(sdk_version reads alpha.7 because the run predates the VERSION bump in this PR.)

https://claude.ai/code/session_01Ckhb622VMbkDVoaKsjVTqg

Summary by CodeRabbit

  • New Features

    • Added SDK usage telemetry for resource method calls, including method, runtime, success status, and duration.
    • Telemetry excludes request and response data, runs best-effort after calls complete, and can be disabled with SUPERSET_TELEMETRY=0 or false.
    • SDK requests now identify the SDK client and version.
  • Breaking Changes

    • Updated low-level SDK request methods to accept structured call descriptors.
  • Documentation

    • Added telemetry details to the SDK reference documentation.
  • Chores

    • Updated the SDK version to 0.0.1-alpha.12.