feat(apify): delete the scrape-notification email type (both senders)
Deletes the automated "new posts" notification entirely. Scraping continues; it just stops emailing.
Tracking issue: recoupable/chat#1955
Why
Scraping a social profile emailed every owner of every artist scraped, one email per artist. On a real label roster that is a flood — measured before this PR: 109 emails to a single recipient over 7 days, peaking at 50 in one day, which is what produced the customer complaint. There was no opt-out, no feature flag, and no per-account preference anywhere in the path.
The feature was built for an account holding a handful of artists. It does not survive a 67-artist roster, and the roster in question is planning 3,000+.
Two senders, deleted together
| Sender | Fired from | For |
|---|---|---|
digest/maybeSendScrapeDigest | apifyWebhookHandler.ts:42 | every scrape batch with new posts |
sendApifyWebhookEmail | handleInstagramProfileScraperResults.ts:112 | non-batch Instagram runs |
They had to go in one commit. Removing only the batched digest would have left customers receiving the legacy single-platform alert, which looks like a fix and isn't. They were also coupled: the legacy sender imported renderScrapeDigestHtml and extractPostsFromDatasetItems from the digest module, so deleting the digest first would have broken that path rather than removed it.
What this does not touch
Scraping, post persistence, Instagram follow-up comment scrapes, and the POST /api/socials/{id}/scrape contract are all unchanged. Verified the digest extractors (extractPostsFromDatasetItems, extractInstagramPosts, extractTiktokPosts, extractTwitterPosts, buildPostStats) were email-only — nothing in the persistence path imported them.
No docs change. The notification behaviour was documented nowhere: POST /api/socials/{id}/scrape describes only run metadata and polling, and no reference page mentions notifications. There is no contract to retract, which is worth noting on its own — this was an undocumented side effect that emailed customers.
Kept deliberately: the scrape-run audit trail
apify_scraper_runs registration (upsertApifyScraperRuns) and completion (updateApifyScraperRun) stay. Together they record which scrapes ran, when they finished, and what was genuinely new — useful for debugging independently of email, and reconstructing this incident without leaning on it cost real time. Only the two select* helpers, read exclusively by the digest, are deleted. No migration; dropping the table would be a schema change with no upside today.
Tests assert the invariant, not the implementation
The new assertions mock @/lib/emails/sendEmail and check sendEmailWithResend is never called, rather than asserting the deleted modules aren't invoked. That survives future refactors: the property being protected is "a scrape sends no email", not "this particular function isn't called."
The Instagram test was a genuine RED → GREEN: with the real sender still wired up it failed with expected "spy" to not be called at all, but actually been called 1 times.
| Check | Result |
|---|---|
| Full suite | 4496 passed / 815 files |
grep -rn "sendEmailWithResend" lib/apify | nothing |
eslint on touched paths | clean |
tsc --noEmit errors in touched files | 0 |
| Diff | 33 files, +46 / −1085 |
Accepted tradeoff
This removes the only proactive "something happened" notification in the product. After this, nothing tells a customer a new post appeared; they see it by opening the app or in a scheduled report. That is a capability going away, not only a bug being fixed, taken deliberately on the evidence that the sole recorded feedback is a complaint. If proactive notifications are wanted later they should be designed for roster scale from the start (one digest per scrape session, not per artist) rather than revived from this code.
Preview verification to follow on this PR.
🤖 Generated with Claude Code
Summary by cubic
Stops all scrape-notification emails; scrapes still persist posts and record run history. Previously we sent per-artist alerts and a batched digest; now we send no email to prevent roster-scale floods (chat#1955).
- Removed senders:
lib/apify/digest/*(batched digest) andlib/apify/sendApifyWebhookEmail.ts(legacy IG alert), plusselectApifyScraperRun(s)helpers used only by the digest. apifyWebhookHandlerstill validates, routes, and callsupdateApifyScraperRun, but no longer callsmaybeSendScrapeDigest.instagram/handleInstagramProfileScraperResultsstill persists posts/socials and queues follow-ups; it resolves account emails but sends none and no longer returnssentEmails.- Tests assert the invariant at the email boundary:
sendEmailWithResendis never called for scrape webhooks or IG runs that find new posts.
Rollout
- No migrations or flags;
apify_scraper_runstable and write path are kept for audit. - Public API and scraping behavior unchanged (
POST /api/socials/{id}/scrape). - This removes our only proactive “new post” notification; no compensating docs change needed since it was undocumented.
Summary by CodeRabbit
- Changes
- Removed scrape digest email notifications and legacy webhook email delivery.
- Scrape completion now records run updates and newly discovered post URLs without sending notifications.
- Instagram processing continues to record new posts and perform follow-up scraping, with non-blocking warning handling.
- Removed unused digest formatting, extraction, recipient, rendering, and delivery functionality.