PR #1055 - basecamp/basecamp-sdk

Piece D of the bc3 → SDK → CLI/MCP pipeline: a report of which SDK operations a downstream repository reaches, for the bump PRs piece C opens in basecamp-cli and basecamp-mcp-server.

go run ./scripts/surface-coverage -sdk v0.30.0 -repo ../basecamp-cli

prints Markdown for a PR body. -format json prints the same report as JSON, and -json FILE writes it alongside. The SDK comes from the module proxy (a tag, main or latest), or from a checkout with -sdk-dir. -ref origin/main reads a git ref of the repository through git archive without touching its working tree. make surface-coverage REPO=../basecamp-cli [SDK=v0.29.1] wraps it.

For C: inputs are the SDK version and the downstream path. Run it against the repo's default branch (-ref origin/main) so coverage and stale references are typed against the code as it is, even when the bump branch doesn't compile yet. From this checkout today, or as go run github.com/basecamp/basecamp-sdk/scripts/surface-coverage@main … once this is on main: the nested module has no version on main yet, so @<branch sha> fails its latest lookup until then.

What it reports

SDK operations the repository does not reach, grouped by tag
references to SDK surface the release doesn't have: a symbol the release doesn't declare, or one that performed an operation the release removed
calls that don't compile against the pinned SDK (type errors naming an SDK symbol)
the delta against the previous Go module release (the newest go/v* tag before it): operations added and not reached, and operations reached through the previous release's SDK and not through this one's
raw HTTP calls whose path matches no operation, or is computed at run time

How coverage is derived: from the code, on both sides

Neither repo keeps a list this could read. The CLI's API-COVERAGE.md is edited by hand, and is wrong in at least one row (below). The MCP server's coverage_test.go checks services, not methods, and its domains.json maps tool names, several of which aren't operationIds. Both sides are type-checked from source with go/types, stdlib only, with imports from go list -export data.

SDK, at the reported version. Each function and method under go/pkg/... is mapped to the operations it performs: the generated-client entry points it references (CreateTodoWithBodyWithResponse is CreateTodo), and the raw Get/Post/Put/Delete calls whose path matches an operation's route. The mapping follows the SDK's internal calls, so TodosService.Update performs GetTodo and ReplaceTodo. It stops at another service's public methods, so Recordings().Summarize doesn't count as covering the 29 reads it makes on the caller's behalf. The operation list is the embedded go/pkg/basecamp/catalog/catalog.json, which make catalog-check holds to openapi.json: 287 operations at v0.29.1 in both.
Downstream. Only the repository's own module is analyzed: a go.work that uses a local SDK checkout doesn't turn the SDK's wrappers into the repository's coverage. Every reference to an SDK function or method in the shipped packages counts: ./cmd/basecamp and its imports for the CLI, ./cmd/basecamp-mcp for the MCP server, tests excluded. References are typed against the SDK version the repo's go.mod pins. A method value counts as a call. So does a call through an interface of the repository's own, named or written inline, when an SDK value is handed to it and only one SDK method can answer the call: embeds create reaches EmbedsService.Create only through embedClient. A raw call's literal, concatenated or Sprintf path is matched against the routes, with constant Sprintf arguments kept as text, and with or without a leading account ID. A symbol the reported SDK doesn't declare is stale even when its name or path still spells an operation. A working-tree run is labeled <sha>-dirty when the tree differs from HEAD, and a dirty local SDK checkout <release>-dirty. Files that import "C" are read through go list -compiled. A -pkg pattern that loads nothing fails the run.

make surface-coverage-test (now a CI step in the Go job) also runs the SDK half against this checkout and requires every catalog operation to have an exported entry point the call graph can see. If a future wrapper idiom hides operations from the graph, that SDK PR fails, so the downstream reports don't quietly count everything as unreached.

Current numbers

The latest tag and main are the same commit today: go/v0.29.1 = 0b66bbc3.

	basecamp-cli `b793a2ff` (pins v0.29.1)	basecamp-mcp-server `769635e8` (pins v0.27.1)
SDK v0.29.1 operations	287	287
Reached	246 (85%)	245 (85%)
Not reached	41	42
New since v0.28.1, not reached	0 of 0	0 of 0
Stale references	0	0
Raw calls not matched to an operation	21 (20 computed)	2 (1 computed)

The previous Go module release is v0.28.1, because v0.29.0 was tagged v0.29.0 with no go/v0.29.0, so the module proxy never had it. Against v0.28.0, the MCP server shows its two newly unreached wallpaper operations under "New in v0.28.0".

Parity findings, filed as cards by area (listed below):

CLI. It doesn't reach timesheet entry create/show/update/delete, though API-COVERAGE.md marks timesheets ✅ "list, show, create, update, delete". The CLI only calls the three report reads. It also doesn't reach the template library and templatifications, project stacks (/stacks, the SDK's Folders), ArchiveProject/UnarchiveProject (it archives through Recordings().Archive), recent projects, my preferences, cloud-file and Google-doc CRUD, the Campfire uploads list, or GetQuestionnaire. Chatbots and client approvals, correspondences and replies are out of scope by the CLI's own note. Card-column subscribe is deliberately not spelled twice.
MCP. It doesn't reach Subtasks, Pings, Embeds or bubble-up create/delete, which are exactly the services coverage_test.go excludes. It also misses method-level gaps that service-level test can't see: archive/unarchive, wallpaper and visit for projects, spotlight and filing for recordings, the template library copies, client enablement, UpdateCampfireLine, UpdateBoost, CreateTool, CreateUploadVersion, UpdateAccountLogo, and the event feed.

The CLI's basecamp mcp dispatches every operation in its vendored model generically. Those calls are the computed-path raw calls; they aren't commands, and the report doesn't count them.

Cards from this run

I filed one card per feature area for each surface's gaps, in Triage, after checking titles against each board. CLI gaps are on the CLIs board:

MCP gaps belong in the MCP project. Three are new there, and each card covering a gap that both surfaces share links to the other. The other twelve were filed on the CLIs board first and are being moved there (their links above still point at the CLIs board):

No card was filed where the board already had one: stacks (the SDK's Folders) are on Unsupported: Stacks, recent projects on Unsupported: Recent projects, and cloud files and Google Docs on [CLI] Docs & Files listings omit cloud files and Google Docs. None was filed either where the CLI records an exclusion with its reason: Clientside approvals, correspondences and replies (API-COVERAGE.md#L279), and card-column subscribe (#L232).

Limits, stated

The tool follows the common spellings and errs toward a visible gap: what it can't follow shows as unreached, or as an unmatched raw call. A spelling it doesn't follow never turns into coverage that isn't there.

Interface bindings follow an SDK value one step, from arguments, assignments, typed vars, returns, conversions, and struct, slice and map values, into an interface the repository declares or writes inline. They don't follow:
- a value wrapped first in a struct of the repo's own
- an interface-typed map key
- a channel send
- a generic function constrained by an interface
A call through an interface handed SDK values with different answering methods is credited to none and listed as ambiguous. A value that doesn't satisfy the interface binds nothing. Neither repo has any of these today.
A raw call whose path comes from a variable or a helper is listed, not matched; in the CLI those are api, show, the dock-tool lookups and the basecamp mcp dispatcher. So is a raw helper called as a method expression, and a Sprintf format with explicit argument indexes or * widths. A Client path must carry the account; an AccountClient path may leave it out.
Calls only through the grouped generated client (generated.Client.Todos().List) aren't mapped. Neither repo makes any.
A method moved onto an embedded type between releases would read as stale. Stale means the symbol is gone. A method that keeps its name but changes its signature is left to the bump PR's build, which reports it with the call site.
The report covers the host's GOOS/GOARCH. Neither downstream has an SDK call in a platform-specific or build-tagged file.

One finding came out of the raw-path check: basecamp-cli's login wizard fetches /my/profile.json through Client with no account (internal/commands/wizard.go:349). That likely misses the profile it greets the user with, and the report lists it as an unmatched raw call.

Tests

The fixtures are a two-release fake SDK (testdata/sdk-v1, sdk-v2; their generated-client stand-ins are kept outside any generated/ directory and placed at pkg/generated in a temp tree by TestMain) and a downstream module pinned to v1 through a replace. They exercise direct calls, method values, an interface seam, an embedded-interface seam, a cross-service composite, literal, concatenated, Sprintf and computed raw paths, a test file and an unshipped package. The delta has added, removed, newly unreached and lost-coverage operations, plus a removed method. Unit tests cover path patterns, route matching (literal beats parameter, a computed segment never fills a fixed word, ties stay unresolved) and semver/pre-release ordering. An end-to-end test runs -ref HEAD against a working tree that differs from the commit.

I applied 52 mutations to the "not reached" detection, the stale and delta logic, the call-site diagnostics, the loading, the labeling and the output, and the suite kills all 52. I ran them again on the final head. The mutations:

coverage itself: not recording it, dropping interface expansion, dropping embedded interfaces, dropping inline interfaces, crediting an ambiguous seam, missing an aliased SDK pointer, losing a generated method behind a seam, binding foreign interfaces, binding a module that merely shares a path prefix, binding the empty interface, binding a value that doesn't satisfy the interface, following composites, not following internal calls, not stripping WithBody, dropping direct generated-client calls
which code counts: counting other workspace modules, counting dependencies, counting test files, dropping -compiled, skipping compiled files without a .go extension (cgo output), accepting a pattern that loads nothing, accepting a wildcard that matches nothing
raw paths: wildcarding a constant Sprintf argument (string and integer), reading indexed formats, refusing a format over a [ or * in its text, treating an absolute AccountClient URL as account-relative, taking a query URL for an absolute one, reading a scheme from a query, dropping matching, letting a Client path leave out the account, requiring the account of an AccountClient path, ignoring the SDK's own raw calls, letting a computed segment fill a literal, dropping the literal-account retry, guessing on ties (two variants), concatenation with no literal
stale and delta: accepting undeclared symbols, taking a removed helper's operations from its first call only, dropping stale detection, calling every unreached operation new, dropping lost coverage
call-site diagnostics: missing-symbol detection by package only, by type only, and on any error
labeling and output: ignoring -ref, never or always marking a tree dirty, not marking a dirty SDK checkout, rounding the percentage up, emitting null lists

I also mutated the real CLI. Removing the one Timesheet().Report call made GetTimesheetReport, and nothing else, newly unreached. Renaming a call to a nonexistent method listed it under calls that don't compile.