fix(cli): report each command once per day instead of on every run

Merged
SA
2mo ago
+101-1

Problem

cli_command_invoked is about 13.3M events a week, roughly 60% of all PostHog events (Production project, last 7 days). 24 users send 80% of it. They are agents and dashboards that run the CLI every few seconds:

  • terminals list: 5.8M
  • terminals read: 3.7M
  • workspaces list: 1.2M
  • status: 0.47M

For example, one open-source fleet dashboard built on the CLI runs terminals list and terminals read every 2.5 s. Each event is also its own analytics.captureEvent call to the API.

Change

trackCommandInvoked now reports each command at most once per UTC day. It stores what it has already reported in $SUPERSET_HOME_DIR/reported-commands.json and skips both the event and the API call for a repeat. If the file cannot be written, the CLI reports as before.

  • Still correct: CLI DAU/WAU, and users per command (the "CLI — top commands" insight now reads as adoption).
  • Gone: raw call counts. "CLI — events per day" (mDPHNp0p) stops meaning anything and should be deleted. I have not changed it.
  • Timing: this only affects new CLI builds. Installed versions keep sending until they update. The heaviest users are on 1.30–1.31.

Verification

  • New analytics.test.ts covers repeats, separate commands, the next UTC day, and an unwritable file. The CLI suite passes (303/303) and typecheck is clean.
  • Dev CLI against a local listener: three runs of superset auth whoami sent one POST /api/trpc/analytics.captureEvent. The other two runs sent only their own calls.
  • Dev CLI against prod: three runs of auth whoami produced one cli_command_invoked event (cli_version 0.0.0-dev) in PostHog.

Summary by cubic

Reduces CLI telemetry volume by reporting each command at most once per UTC day instead of on every run, cutting ~13.3M weekly cli_command_invoked events (about 60% of all PostHog events) from agents and dashboards that run the CLI every few seconds, and requires no migration—only new CLI builds.

  • Stores reported commands in $SUPERSET_HOME_DIR/reported-commands.json and skips both the event and the API call for repeats; falls back to reporting every run if the file can't be written.
  • Ignores a malformed or unreadable reported-commands.json file instead of throwing, so one bad file won't break a command.
  • DAU/WAU and users-per-command insights stay correct, while raw call counts go away.

Summary by CodeRabbit

  • Analytics
    • Command usage is reported at most once per command per UTC day. Different commands are tracked separately, and reporting becomes available again the following UTC day. If reporting data is unavailable or cannot be updated, command usage reporting continues.