refactor(songstats-backfill): remove local quota ledger, Songstats is rate authority (chat#1797 — bullet 2)

Merged
SW
4mo ago
+42-236

What

Bullet 2 of recoupable/chat#1797: remove the local quota ledger + budget gate. Songstats is the source of truth for rate limits, so there's no value in mirroring a quota locally (it drifts, and the prod 429 storm showed it was filling with phantom 429 "hits" → premature drain halt).

Backfill now relies entirely on the per-track bounded exponential backoff from PR1 (#673). When a track is still rate-limited past the backoff bound, the step defers (leaves the row pending) and the workflow stops the drain — the rest stays pending for the next run.

Changes

  • Remove getBackfillBudgetStep + test
  • Remove insertSongstatsQuotaLedger / selectSongstatsQuotaSpent + tests
  • backfillTrackStep: no ledger write; returns { ok, deferred? }
  • songstatsBackfillWorkflow: drop the budget gate; drain-until-empty-or-deferred

The songstats_quota_ledger table itself is dropped in a follow-up database migration PR (sequenced after this code stops reading/writing it).

Stacked PR

Based on fix/songstats-backfill-backoff-pacing (#673), not test. Review/merge #673 first; this will retarget to test automatically once #673 merges.

Tests

20 passing across app/workflows + lib/songstats; full regression green. tsc + lint clean.

Closes bullet 2 of recoupable/chat#1797.

Summary by cubic

Removed the local Songstats quota ledger and budget gate. Backfill now defers to Songstats rate limits with per-track bounded backoff; it stops on the first deferred track and releases the rest of the claimed batch to pending. Implements chat#1797 bullet 2.

  • Refactors
    • Deleted all songstats_quota_ledger code/tests and getBackfillBudgetStep.
    • backfillTrackStep: removed ledger writes and hitsSpent; returns { ok, deferred? }; defers by leaving the row pending.
    • songstatsBackfillWorkflow: no budget gate; claims fixed-size batches and drains until empty or first defer; releases unprocessed rows to pending; returns { backfilled, terminal, deferred }.
    • Trimmed workflow JSDoc and removed stray screenshots.