Migrate research provider to SongStats

Merged
SI
5mo ago
+1353-2300

Summary

  • Make SongStats the default research provider via a provider router while preserving Chartmetric behind RESEARCH_PROVIDER=chartmetric.
  • Add SongStats Enterprise API fetcher and compatibility mapper for supported artist, track, search, lookup, stats, audience, catalog, activities, and playlist paths.
  • Keep unsupported Chartmetric-only paths explicit with a 501 compatibility response instead of fabricating unavailable SongStats data.

Verification

  • pnpm test lib/research lib/songstats lib/chartmetric passed: 42 files, 195 tests.
  • pnpm lint:check passed with two unrelated warnings in generated Workflow route files.
  • env STRIPE_SK=sk_test_dummy SLACK_BOT_TOKEN=dummy SLACK_SIGNING_SECRET=dummy SLACK_CHAT_BOT_TOKEN=dummy SLACK_CHAT_SIGNING_SECRET=dummy pnpm build passed.
  • pnpm build without dummy env fails because local required runtime env vars are absent (STRIPE_SK, Slack bot/signing vars).
  • pnpm exec tsc --noEmit still fails on existing repo-wide fixture/type drift outside research files.

Summary by cubic

Switch research to SongStats as the sole provider and remove Chartmetric. Routes use a SongStats mapping layer; metrics and playlists call provider endpoints directly with longer timeouts.

  • New Features

    • Added fetchSongstats (Enterprise API with apikey, 10s default timeout; 50s for slow stats/historic_stats and artists/top_playlists; graceful 500/504) and fetchSongstatsResearch to map research paths and normalize results.
    • Metrics via getResearchMetrics (resolve artist or use id, call /artist/:id/stat/:source, deduct credits); /api/research/metrics sets maxDuration=60.
    • Playlists use SongStats artists/top_playlists with status→scope mapping and flattened placements; /api/research/playlists sets maxDuration=60.
    • Search hits SongStats and returns normalized artists/tracks/labels and related artists; dropped beta and platforms.
    • Validation accepts provider IDs: artist endpoints also accept id; /research/lookup accepts a Spotify artist URL or spotifyId.
    • Metric sources map to SongStats IDs and include radio and sxm. Track ID extraction simplified in resolveTrack.

  • Migration

    • Set SONGSTATS_API_KEY (legacy SongStats_API is not supported). Removed all Chartmetric env, code, and tests.
    • Removed routes not supported by SongStats: /research/charts, /cities, /curator, /discover, /festivals, /genres, /instagram-posts, /playlist, /radio, /venues, /rank.
    • Update clients: use provider IDs (non‑numeric allowed) for artists/tracks; pass artist or id on artist-scoped endpoints; use SongStats search; stop sending beta/platforms; drop Chartmetric-only playlist filters (keep platform, status, limit).

Summary by CodeRabbit

  • Removed Features

    • Removed research endpoints: /api/research/charts, /cities, /curator, /discover, /festivals, /genres, /instagram-posts, /playlist, /radio, /rank, /venues.

  • Modified Features

    • Research identifiers are provider-agnostic (non-numeric) and artist lookups accept Spotify URL or direct Spotify ID.
    • Search results now surface labels and suggestions when primary categories are absent.
    • Metrics and Playlists routes allow extended timeouts.
    • Metrics sources expanded to include radio and sxm.