feat(review): telemetry for review feature usage
Issue for this PR
Closes #1065
Type of change
- Bug fix
- New feature
- Refactor / code improvement
- Documentation
What does this PR do?
The review feature emitted no telemetry at all — no Telemetry reference in cli/cmd/review.ts or the engine, and no session on the CLI path, so not even session_start. We couldn't tell how often review runs, who runs it, what it finds, or whether results get published.
Adds two events from one helper shared by both callers of the engine — the review command and the dbt_pr_review tool — separated by an invocation field. Instrumenting only the command would have missed every review an agent runs through the tool.
Caller attribution needed no code. toAppInsightsEnvelopes already seeds source from Flag.ALTIMATE_CLI_CLIENT, and an event only overrides that by declaring its own source field. These events declare none, so anything exporting ALTIMATE_CLI_CLIENT is attributed for free. That let me drop an earlier design carrying a global envelope property, a per-command entrypoint registry, a reviewPullRequest() signature change and typed errors through the engine.
Four choices that aren't obvious from the diff:
- Publication is a separate event. It happens after the review is computed and can partially succeed, so it can't be a field on
review_run, and a failed post must not mark a computed review as failed. partialcovers every degraded post state.PostResultcan't distinguish them —postErrorisn't cleared when the retry succeeds, and an inline fallback coexists with a realreviewId.- Two failure reasons plus a fallback. The engine degrades rather than throwing for missing manifests, dispatcher failures and the AI lane, so buckets for those would never fire.
by_categoryis zero-filled from the 14-value enum and drops unknown keys, so "never fired" differs from "not possible" and a malformed category can't mint a new dimension.
Enums, booleans and counts only — review findings are about customer schema.
How did you verify your code works?
- 13 unit tests. Mutation-checked rather than assumed: adding a
sourcefield to the event fails the attribution test, and removing the zero-fill fails two others. - One opt-in e2e (
ALTIMATE_E2E=1 bun test test/e2e/, ~13s) that runs a realaltimate-code reviewprocess against a real git repo, with telemetry pointed at a local HTTP sink. Spawned with onlyALTIMATE_CLI_CLIENTset, the arriving envelope carriedsource=plugin:claude-code— which is the one claim the whole design rests on and can't be shown in-process. - That run also caught something I'd documented wrongly: a repo with a
dbt_project.ymlbut no compiled manifest reportsdegraded=true, sodegradedis a fidelity flag, not a "no warehouse" flag. Docs corrected. - Privacy asserted, not assumed: a unit test checks no schema identifier appears in the serialized event, and the e2e greps the whole payload for the fixture's file name, changed SQL and repo path.
tsgo --noEmitclean onpackages/opencode; 162 telemetry tests passing.
Screenshots / recordings
N/A — no UI change.
Checklist
- I have tested my changes locally
- I have not included unrelated changes in this PR
🤖 Generated with Claude Code
https://claude.ai/code/session_018SLUQF3xgZHsGZHSjxe7vb
Summary by CodeRabbit
-
New Features
- Added telemetry for review execution and publication outcomes across CLI and tool-based reviews.
- Captures completion, failure, duration, verdict, quality metrics, and posting results while protecting repository and customer content.
- Preserves existing review output and error behavior when telemetry is unavailable.
-
Documentation
- Documented review telemetry events, payload fields, privacy safeguards, publication statuses, and event correlation.