feat(rules): pause or block teams on high bounce and complaint rates

KM
37m ago
+2139-184

Step 2 of the useSend Guard plan. Rebased onto main on Nov 21. Do not merge without the founder's yes.

Safe by default: with no setting, this runs in alert mode. It only posts to Discord and logs an ALERT. It never pauses, blocks or emails a team until SENDING_RULES_MODE=enforce is set on Railway.

What it does (cloud only)

After every hard bounce or spam complaint, checkSendingHealth checks the team, and the campaign if there is one:

Hard bouncesComplaints
Pause> 2.5% and at least 5> 0.08% and at least 2
Block> 5% and at least 10> 0.3% and at least 3
  • Team: rolling 7 days from DailyEmailUsage. Days up to an admin resume or unblock no longer count.
  • Campaign: its own counters. A bad campaign is paused on its own. If its owner resumes it, only the block level can stop it again.
  • Spam trap: a hard bounce whose diagnostic names a spam trap or honeypot blocks the team at once.
  • Pause: sets Team.sendingPausedAt, pauses running and scheduled campaigns, refreshes the team cache, logs a TeamAction, posts to Discord, and emails the team with a short, plain explanation. New emails fail with EMAIL_SENDING_PAUSED. Campaigns can't be resumed while the team is paused or blocked.
  • Block: the same, but sets isBlocked.
  • Concurrency: conditional updateMany (where isBlocked = false / sendingPausedAt = null) means a burst of events acts once.
  • Fail-safe: a failing check is logged and never breaks SES event processing.

Mode: SENDING_RULES_MODE

  • alert (default): only logs an ALERT and posts [alert only] Would pause… to Discord, at most once a day per team or campaign.
  • enforce: acts at once. Pauses or blocks, emails the team, and posts to Discord. Only when set explicitly.
  • off: disabled.

Dry run on the last 7 days of production data (Nov 15–21, 32 sending teams, 26k sends): one team (#104, 5 of 9 emails hard-bounced) would have reached the pause level, and none the block level. In alert mode that's one Discord message.

Avoiding false positives

  • Matched sends and outcomes, per email: when SES accepts an email, one transaction counts it in the team's SendingCohort row for today (and Campaign.acceptedSends) and stores that day on the email (Email.cohortDate). A hard bounce or complaint counts only if its email carries that mark, and it's filed under exactly that day. Emails without the mark don't count: ones sent before this deploy, by an old worker during the overlap, or whose count failed. Team rates compare the last 7 days of counted sends with the outcomes of those same emails, whenever they arrive.
  • Duplicate SES events: a hard bounce or complaint is claimed atomically per email with Redis SET NX. A repeated event counts once for the rules and the dashboards, and the rules don't re-run on it. If Redis fails, the event isn't counted, because undercounting can't wrongly pause anyone.
  • Admin resume or unblock: the rules store which campaigns hadn't sent yet. Campaigns that had already sent aren't judged on their totals again, and the team window restarts the day after the reset. Drafts are judged in full.
  • Stale checks: before acting, the rules lock the team row (the same lock the admin update takes) and confirm no new reset happened since the check started.
  • Campaigns: campaigns are judged the same way, on Campaign.acceptedSends and the hard bounces and complaints of those same marked emails (acceptedHardBounced, acceptedComplained). The SES-event counters that feed the dashboards aren't used by the rules.
  • Known undercount (safe direction): an outcome processed before its send transaction commits finds no mark and isn't counted.

Admin

  • Teams page: "Sending paused" switch, a badge, and the last 10 actions (who, what, why).
  • Actions logged: admin block, unblock, pause and resume are recorded as TeamAction with actor admin:<email>, in the same transaction as the change.

Schema

TeamAction table (+ TeamActionType enum), SendingCohort table, Team.sendingPausedAt, Email.cohortDate (nullable, so adding it is instant) and Campaign.acceptedSends / acceptedHardBounced / acceptedComplained (constant defaults, so also instant). Migration 20261121120000_sending_rules_and_team_actions (re-dated so it sorts after the Nov 17 migrations) is generated by prisma migrate diff, additive only, and not run anywhere. It sets lock_timeout = 10s, so it can't queue behind a long query and hold up writes to Email. Railway runs db:migrate-deploy before next start; if it fails, the old deployment keeps serving (health check).

Tests

  • New sending-rules-service.unit.test.ts (20):
    • thresholds and minimum counts
    • spam-trap detection
    • pause, block and spam-trap actions
    • acting only once
    • alert mode (once a day, no changes)
    • campaign pause, and no re-pause after a resume
    • the window resetting after an admin resume
    • off and self-hosted
  • ses-hook-parser: runs on hard bounces, flags spam traps, survives a failing check, doesn't run on opens.
  • limit-service: a paused team can't send.
  • resumeCampaign: refuses while the team is paused or blocked, before touching the batch.
  • Typecheck clean; unit (416), tRPC (23) and API (50) suites pass; production build passes.

Summary by cubic

Adds automatic pause and block rules for teams on high bounce or complaint rates, so bad lists don't jeopardize our SES account (cloud only). The check runs after every hard bounce or spam complaint and is gated by SENDING_RULES_MODE: enforce pauses or blocks, alert (default) only logs and posts to Discord, off disables it.

Rules and effects

  • Pause at >2.5% hard bounces (min 5) or >0.08% complaints (min 2); block at >5% (min 10) or >0.3% (min 3) over 7 days. A spam trap bounce blocks immediately.
  • SendingCohort files each send and its bounces and complaints under the day the email was sent (Email.cohortDate); campaigns are judged on their counted sends (Campaign.acceptedSends), not dashboard SES-event counters.
  • Duplicate SES bounces and complaints count once via a Redis claim, which expires after 13 days; outcomes are only counted for emails sent within that window, so a late replay can't count them twice.
  • Sends count as at least the delivered plus bounced. Dashboard usage counters write in one transaction: a failed write releases the claim so a replay can retry, while a failed rules-counter update keeps it so the outcome can't count twice.
  • Pausing sets Team.sendingPausedAt, stops running and scheduled campaigns, fails new sends with EMAIL_SENDING_PAUSED, and shows a banner in the app. Retrying a campaign is refused while paused or blocked, and the resume re-checks the team under the same row lock so a pause landing in between can't be overwritten. A bad campaign is paused on its own; after an admin resume, only the block level pauses it again.
  • Admin resume clears the pause, restarts the team window the day after, and exempts campaigns that had already sent from re-judgment; drafts are judged in full.
  • Adds TeamAction table, and a sending-paused switch, badge, and action log on the teams admin page.
  • Conditional updateMany guards against acting twice on event bursts; the rules lock the team row and confirm no admin reset happened mid-check. A failing check only logs; Discord posts and team emails run in the background.
  • Migration 20261121120000_sending_rules_and_team_actions is additive-only and sets a 3s lock timeout so it fails fast rather than queue behind a long query; self-hosted installs skip all rules and cohort tracking.
  • Waitlisted users are redirected to the waitlist page showing their saved application, which can be updated; the admin waitlist page shows when it was last updated.

Summary by CodeRabbit

  • New Features
    • Added sending-health monitoring for hard bounces, complaints, and spam-trap reports, with alerts or automatic sending pauses and blocks.
    • Added admin controls to pause or resume team sending, view recent team actions, and see when sending is paused.
    • Added notices in the dashboard and upgrade flow when sending is paused.
  • Bug Fixes
    • Prevented campaigns from resuming when a team is blocked or sending is paused.
    • Waitlisted users are now directed to the waitlist page, and application status shows when it was last updated.

Ask Polylane for an impact assessment of this pull request:

Assess with Polylane