feat: Apify-first Spotify play counts for GET /api/research/track/stats

Merged
SW
4mo ago
+1357-390

What

First api slice of recoupable/chat#1791: GET /api/research/track/stats goes Apify-first for Spotify, fulfilling the contract merged in recoupable/docs#238 against the store merged in recoupable/database#32.

Behavior (route + handler signatures unchanged):

  • Request carries isrc + source including spotify → the Spotify entry is served from song_measurements. Fresh capture (TTL SPOTIFY_PLAYCOUNT_TTL_HOURS, default 24h) → store read, zero vendor calls. Stale/missing → one actor call for the whole album (beatanalytics~spotify-play-count-scraper; pricing is per album URL), writing every sibling track with an identifier mapping; raw_ref = actor run id.
  • Actor failure → degrade to the stale capture; nothing stored → automatic full Songstats fallback. Songstats keeps serving all non-Spotify sources.
  • Every stat entry now carries data_source provenance (apify_spotify_playcount | songstats) + captured_at on store-served entries, per the contract.
  • Songstats is out of the Spotify request path whenever the store can answer — quota spend on Spotify current-stats drops to zero for measured tracks.
  • Also in this PR: database.types.ts regenerated post database#31+#32 (proves nothing was typed against the 14 dropped tables — tsc error count unchanged at 198 pre-existing, all in unrelated test files).

New units (each RED→GREEN, mocked like siblings)

UnitTests
lib/supabase/song_measurements/selectLatestSongMeasurement3
lib/supabase/song_measurements/insertSongMeasurements (fetch-once upsert, ignoreDuplicates)3
lib/supabase/song_identifiers/selectSongIdentifiers2
lib/supabase/song_identifiers/selectSongsByIdentifierValues (reverse lookup)3
lib/apify/spotify/fetchSpotifyAlbumPlayCounts (actor wrapper)3
lib/research/playcounts/getSpotifyStatFromStore (TTL → refresh → degrade → fallback)5
lib/research/getResearchTrackStats (routing + merge + provenance labels)5 rewritten

Full regression: 340 tests / 74 files pass; eslint + prettier clean; tsc delta zero.

Verification plan (preview)

  1. Fallback path (live now): stats request for any ISRC → Songstats-served entries each carry data_source: "songstats"; 400/401/404 paths unchanged.
  2. Apify path needs song_identifiers seed rows (album_id + track_id mappings) — the store is empty until the snapshots endpoint (next PR) populates it. Will verify with a seeded Rostrum track and post results as PR comments.

Note: requires APIFY_TOKEN in the api Vercel env (already present — the social scrapers use it).

Merge order

docs#238 ✅ → database#31/#32 ✅ → this → historic-stats stitching → backfill workflow → snapshots+deltas.

🤖 Generated with Claude Code


Summary by cubic

Switches Spotify play counts in GET /api/research/track/stats to an Apify‑first, store‑backed path with OCP routing over the Songstats passthrough; eliminates Songstats usage for Spotify on measured tracks and adds clear data_source provenance — part of recoupable/chat#1791.

  • New Features

    • Serve Spotify stats from song_measurements with TTL (SPOTIFY_PLAYCOUNT_TTL_HOURS, default 24h): fresh → store read; stale/missing → single Apify actor run per album (beatanalytics~spotify-play-count-scraper, album ids deduped), writing mapped sibling tracks with raw_ref = run id.
    • Actor failure → return stale; empty store → automatic Songstats fallback. Non-Spotify sources stay on Songstats. Each stat includes data_source (apify_spotify_playcount | songstats); store-served also include captured_at. Credits deducted on successful store reads. Requires APIFY_TOKEN. No API changes.
  • Refactors

    • True OCP routing: new getTrackStatsApifyFirst layered over the untouched getResearchTrackStats passthrough; handler now calls it.
    • Parse once at the orchestrator boundary via trackStatsPayloadSchema; labelSongstatsProvenance is a pure typed transform, and appending the store stat is now inlined (KISS). Unexpected Songstats payloads log a warning and pass through unlabeled (no response failures).
    • Supabase accessors: selectSongIdentifiers (forward/reverse), selectSongMeasurements (newest‑first, optional limit), upsertSongMeasurements (append‑only with ignoreDuplicates). Ignore supabase/.temp/; regenerate and reformat types/database.types.ts.

Summary by CodeRabbit

  • New Features
    • Added Spotify album and track playcount data retrieval
    • Integrated Spotify playcount metrics into track statistics queries
    • Implemented measurement caching with automatic freshness detection for improved performance