feat: Apify-first Spotify play counts for GET /api/research/track/stats
What
First api slice of recoupable/chat#1791: GET /api/research/track/stats goes Apify-first for Spotify, fulfilling the contract merged in recoupable/docs#238 against the store merged in recoupable/database#32.
Behavior (route + handler signatures unchanged):
- Request carries
isrc+sourceincludingspotify→ the Spotify entry is served fromsong_measurements. Fresh capture (TTLSPOTIFY_PLAYCOUNT_TTL_HOURS, default 24h) → store read, zero vendor calls. Stale/missing → one actor call for the whole album (beatanalytics~spotify-play-count-scraper; pricing is per album URL), writing every sibling track with an identifier mapping;raw_ref= actor run id. - Actor failure → degrade to the stale capture; nothing stored → automatic full Songstats fallback. Songstats keeps serving all non-Spotify sources.
- Every stat entry now carries
data_sourceprovenance (apify_spotify_playcount|songstats) +captured_aton store-served entries, per the contract. - Songstats is out of the Spotify request path whenever the store can answer — quota spend on Spotify current-stats drops to zero for measured tracks.
- Also in this PR:
database.types.tsregenerated post database#31+#32 (proves nothing was typed against the 14 dropped tables — tsc error count unchanged at 198 pre-existing, all in unrelated test files).
New units (each RED→GREEN, mocked like siblings)
| Unit | Tests |
|---|---|
lib/supabase/song_measurements/selectLatestSongMeasurement | 3 |
lib/supabase/song_measurements/insertSongMeasurements (fetch-once upsert, ignoreDuplicates) | 3 |
lib/supabase/song_identifiers/selectSongIdentifiers | 2 |
lib/supabase/song_identifiers/selectSongsByIdentifierValues (reverse lookup) | 3 |
lib/apify/spotify/fetchSpotifyAlbumPlayCounts (actor wrapper) | 3 |
lib/research/playcounts/getSpotifyStatFromStore (TTL → refresh → degrade → fallback) | 5 |
lib/research/getResearchTrackStats (routing + merge + provenance labels) | 5 rewritten |
Full regression: 340 tests / 74 files pass; eslint + prettier clean; tsc delta zero.
Verification plan (preview)
- Fallback path (live now): stats request for any ISRC → Songstats-served entries each carry
data_source: "songstats"; 400/401/404 paths unchanged. - Apify path needs
song_identifiersseed rows (album_id + track_id mappings) — the store is empty until the snapshots endpoint (next PR) populates it. Will verify with a seeded Rostrum track and post results as PR comments.
Note: requires APIFY_TOKEN in the api Vercel env (already present — the social scrapers use it).
Merge order
docs#238 ✅ → database#31/#32 ✅ → this → historic-stats stitching → backfill workflow → snapshots+deltas.
🤖 Generated with Claude Code
Summary by cubic
Switches Spotify play counts in GET /api/research/track/stats to an Apify‑first, store‑backed path with OCP routing over the Songstats passthrough; eliminates Songstats usage for Spotify on measured tracks and adds clear data_source provenance — part of recoupable/chat#1791.
-
New Features
- Serve Spotify stats from
song_measurementswith TTL (SPOTIFY_PLAYCOUNT_TTL_HOURS, default 24h): fresh → store read; stale/missing → single Apify actor run per album (beatanalytics~spotify-play-count-scraper, album ids deduped), writing mapped sibling tracks withraw_ref= run id. - Actor failure → return stale; empty store → automatic Songstats fallback. Non-Spotify sources stay on Songstats. Each stat includes
data_source(apify_spotify_playcount|songstats); store-served also includecaptured_at. Credits deducted on successful store reads. RequiresAPIFY_TOKEN. No API changes.
- Serve Spotify stats from
-
Refactors
- True OCP routing: new
getTrackStatsApifyFirstlayered over the untouchedgetResearchTrackStatspassthrough; handler now calls it. - Parse once at the orchestrator boundary via
trackStatsPayloadSchema;labelSongstatsProvenanceis a pure typed transform, and appending the store stat is now inlined (KISS). Unexpected Songstats payloads log a warning and pass through unlabeled (no response failures). - Supabase accessors:
selectSongIdentifiers(forward/reverse),selectSongMeasurements(newest‑first, optional limit),upsertSongMeasurements(append‑only withignoreDuplicates). Ignoresupabase/.temp/; regenerate and reformattypes/database.types.ts.
- True OCP routing: new
Summary by CodeRabbit
- New Features
- Added Spotify album and track playcount data retrieval
- Integrated Spotify playcount metrics into track statistics queries
- Implemented measurement caching with automatic freshness detection for improved performance