feat(web): add MCP and API key usage tracking to analytics

Merged
MS
8mo ago
+1113-157

Summary

Analytics: MCP & API usage tracking

  • Move audit event creation from client-side search bar to service functions (search, streamSearch, listRepos, getFileSource, getTree)
  • Add source metadata to audit events to distinguish MCP requests from other API calls
  • Extend analytics SQL query to include new audit actions and display MCP request and API request counts
  • Add two new charts to analytics dashboard: MCP Requests and API Requests

Audit log retention policy

  • Add SOURCEBOT_EE_AUDIT_RETENTION_DAYS environment variable (default: 180) to configure how long audit logs are kept
  • Add AuditLogPruner background job that runs daily and prunes audit records older than the configured retention period, deleting in batches of 10K to avoid long-running transactions
  • Set to 0 to disable pruning and retain logs indefinitely

Analytics UI: retention info

  • Surface the configured retention period and oldest audit record date in the analytics page header
  • Wrap the analytics response in an object ({ rows, retentionDays, oldestRecordDate }) instead of returning a flat array

Documentation updates

  • Audit logs docs: add retention policy section, update audit action types table (removed 4 stale actions, added 11 missing actions), add chat to targetType enum, add source to metadata schema
  • Environment variables docs: add SOURCEBOT_EE_AUDIT_RETENTION_DAYS entry
  • Sizing guide: add audit log storage section with enterprise callout, per-event storage estimate (~350 bytes), projected storage table for various team sizes, and retention policy guidance

Mock data script

  • Update inject-audit-data.ts with 5 mixed-usage user profiles (web-only, web+MCP hybrid, MCP-heavy, API-only, web+API hybrid) with randomized weights
  • Add all new audit actions and source metadata for MCP/API users

Screenshots

image

Retention policy and oldest audit log date displayed in analytics header:

Retention info in analytics header

Test plan

  • Verify web UI searches still fire audit events and appear in analytics
  • Verify MCP tool invocations fire audit events with source='mcp' and appear in MCP Requests chart
  • Verify non-MCP API key usage fires audit events and appears in API Requests chart
  • Verify API users count as active users in analytics
  • Verify retention period and "data since" date display correctly in analytics header
  • Verify audit log pruner runs daily and correctly deletes records older than the configured retention period
  • Verify setting SOURCEBOT_EE_AUDIT_RETENTION_DAYS=0 disables pruning

Linear: SOU-579

šŸ¤– Generated with Claude Code

Summary by CodeRabbit

New Features

  • Audit log retention policy: Automatically prune audit logs older than 180 days (configurable via environment settings; default 180 days, set to 0 to disable).
  • Enhanced analytics: View metrics breakdown by source (web app, MCP, API) for better visibility into usage patterns.
  • Improved audit tracking: Source information now captured for all user actions to distinguish between web UI, API, and other client sources.

Documentation

  • Added audit log retention configuration guide and sizing recommendations for audit log storage.