feat(migrations): digest-batch columns on apify_scraper_runs (batch_id, completed_at, new_post_urls)

Closed
SW
3mo ago
+23-0

Summary

Schema prerequisite for the one-digest-per-scrape design on recoupable/chat#1855 (PR #2 of 6): a roster scrape starts one Apify run per platform, each completing independently by webhook — to send one consolidated email instead of 4+, completions need to find their siblings (batch_id), know when the batch is done (completed_at), and know what each run found (new_post_urls, diffed against existing posts so only genuinely-new content digests).

Nullable columns, no FKs, no backfill — rows predating the migration never digest, matching the table's loose-ids stance from chat#1840. Partial index on batch_id for the sibling lookup. Additive and safe to apply independently of the api PRs (columns sit unused until the aggregation PR lands).

🤖 Generated with Claude Code


Summary by cubic

Add digest-batch columns on apify scraper runs to group per-scrape runs and send one consolidated “new posts” digest. Adds a partial index for fast sibling lookups.

  • Migration
    • Added columns: batch_id (UUID), completed_at (timestamptz), new_post_urls (JSONB); all nullable.
    • Created partial index on batch_id where not null.
    • No FKs and no backfill; existing rows won’t digest. Additive and safe until aggregation logic ships.