feat(web): 90-day-ready email content cleanup: batched, safe on large tables, runs on cloud when enabled

Merged
KM
4d ago
+680-79

Why

Cloud keeps every email's HTML, text, attachments and headers forever. That's 3.7 GB of the 7.1 GB production database, and it grows about 250 MB a month. We're setting a 90-day retention for email content on cloud. Delivery status and events are kept.

This PR changes nothing on its own. The cleanup only runs where EMAIL_CLEANUP_DAYS is set, and on cloud it'll be set only after customers are told. The one thing that applies on deploy is the autovacuum tuning on Email, which changes a table setting and touches no data.

What it does

  • Rule: clear html, text, attachments and headers of emails sent more than EMAIL_CLEANUP_DAYS ago, meaning created and last updated before the cutoff. Sending updates the row, so an email that waited in a queue keeps its content for N days after it went out. SES status updates don't touch updatedAt.
  • Never cleared: scheduled or queued emails, and emails scheduled after the cutoff.
  • Nightly pace: walks forward by createdAt from a saved position, so it never rereads emails it has already looked at. It clears at most 100,000 emails a night, in batches of 500. Each batch is its own transaction with lock_timeout 5s and statement_timeout 60s, there's a 15-minute limit per run, and short pauses between batches.
  • Skipped emails: emails looked at but not clearable yet go into a Redis sorted set, scored by when they become eligible, and are rechecked by id once due. Production Redis is noeviction with no maxmemory, so the set can't be evicted on its own.
  • Untouched fields: doesn't change updatedAt, which the campaign page sorts by.
  • Vacuum: no VACUUM from the app. A migration sets autovacuum_vacuum_scale_factor and autovacuum_analyze_scale_factor to 0.02 on Email, inside a transaction with lock_timeout 10s. That makes autovacuum visit after about 2% of rows change, instead of 20%.
  • Dashboard: the email page explains removed content ("removed N days after sending; status and events are kept") instead of "No content to preview".
  • Self-hosted: the same job replaces the old single unbatched UPDATE. The self-hosting docs are updated.

Tested

  • Real Postgres (cleanup-email-bodies.integration.test.ts, runs in CI):
    • The right emails are cleared, and recent, scheduled, queued and late-sent ones are kept.
    • updatedAt is untouched.
    • A late-sent email is rechecked exactly when due (not at 70 days, cleared at 81 days on a simulated clock).
    • Ties on createdAt across batches are all handled.
    • The cap holds exactly, in both the recheck and scan phases.
    • Removing the updatedAt condition fails 3 tests.
  • Scale: Postgres 16 with 400k production-shaped emails (~4 KB TOASTed bodies).
    • One night of 50k took 23 s and wrote 41–110 MB of WAL.
    • With autovacuum tuned, two nights of 100k left no dead tuples, and the table was 278 MB smaller.
    • The keyset batch query is an Index Scan Backward on createdAt plus an Incremental Sort, about 30 pages per batch.
  • Migrations: apply on a fresh database, and prisma migrate diff (database → schema) is empty.
  • Suites: unit (376) and tRPC (23) tests pass, and the production build passes.
  • Review: Astra reviewed three rounds, and every finding is addressed.

Turning it on (cloud, later)

  1. Customer notice, plus privacy policy and docs updates (drafts to be approved separately).
  2. Set EMAIL_CLEANUP_DAYS=90 on Railway and redeploy.
  3. Watch the first nights: [EmailCleanupJob] logs, dead tuples, WAL and free disk.

The backlog of about 1.2M emails clears in about 12 nights.

Ask Polylane for an impact assessment of this pull request:

Assess with Polylane


Summary by cubic

Rebuilds the email content cleanup so it works on cloud at scale, and explains removed content in the email dashboard.

The old job was a single UPDATE over all old emails and only ran on self-hosted installs. The new job runs wherever EMAIL_CLEANUP_DAYS is set and works in small nightly batches, so nothing changes until that env var is set. The only change that applies on deploy is a migration that tunes autovacuum on Email — it alters a table setting and touches no data.

  • Walks emails forward by createdAt with a saved position, clearing at most 100,000 per night in batches of 500, each batch its own transaction with lock_timeout 5s, statement_timeout 60s, and a 15-minute run limit.
  • Clears html, text, attachments, and headers only for emails created and last updated before the cutoff, so scheduled, queued, and late-sent emails keep their content; delivery status and events are always kept.
  • Emails looked at but not ready yet are tracked in a Redis sorted set (scored by when they become eligible) and rechecked by id once due.
  • Does not touch updatedAt, which the campaign page sorts by.
  • Replaces the app-level VACUUM with autovacuum tuning on Email (autovacuum_vacuum_scale_factor and autovacuum_analyze_scale_factor at 0.02).
  • The email details page explains that content was removed after N days instead of showing "No content to preview", and the self-hosting docs are updated.

Summary by CodeRabbit

  • New Features
    • Email content is cleared after the configured retention period, while delivery status and event history are retained.
    • Scheduled or queued emails keep their content until sending. Older eligible emails may be cleaned up gradually, with a nightly limit.
    • Email previews show a retention notice when content has been removed.
  • Documentation
    • Updated self-hosting guidance with details about retention, cleanup limits, and freed storage.