fix(pipeline): age-gate ancient provider and fresh-enrichment sweep wakes

Merged
GI
7h ago
+404-9

Problem

Maintenance sweeps repeatedly re-enqueue months-old provider-STT and enrichment-fresh obligations ahead of current work. Wave 4 t6 identifies stale provider wakes and ancient enrichment retries as the queue contaminant; increasing dispatch alone does not stop that generator.

Fix

Skip sweep re-wakes strictly older than 14 days, independently configurable per queue. Provider-STT uses durable created_at; enrichment-fresh uses the most recent obligation timestamp (latest_pending_at, with first_pending_at for existing rows) and checks its stored or repaired destination. Fresh coalesced marks advance that timestamp without changing the original acceptance or deadline. Two weeks preserves a normal recovery window while excluding t6's months-old obligations. Missing, zero, negative, or otherwise unknown age metadata remains eligible. Initial enqueue and consumers are unchanged.

Age-skipped rows are left byte-for-byte unchanged: no deletion, parking, token rotation or lease write. They remain durable for a separately bounded backfill. Existing privacy/deletion handling takes precedence over age skipping in both sweeps. Provider-STT advances an in-process document-ID cursor across bounded pages, including skipped rows, and wraps at exhaustion; a full ancient page cannot strand later fresh rows. The existing state single-field index suffices; no index is added. The existing independent 72-hour enrichment parking policy runs first; the new gate applies only to obligations that policy leaves open. Existing parking removes historical rows from the due scan without adding any new retirement or dead-letter behavior. Setting the new gate off restores legacy re-wake admission; the existing parking policy remains in force.

omi_batch_pipeline_sweep_skipped_total{queue,reason="max_age"} counts skipped candidates, not unique rows. Both Cloud Run GMP profiles already retain it; a test pins that coverage. Accessors live in config/policy, with budget ownership and generated registry entries.

Tests

BACKEND_UNIT_TEST_FILE_LIST + bash backend/test.sh via omi-heavy: 321 passed across 7 affected files (provider outbox, sweep age contract, enrichment coalescing, enrichment route, existing enrichment age policy, batch-stage env ratchet, flag budget).

Regressions cover an all-ancient provider page followed by fresh lost-wake work, cursor wrap, a fresh coalesced mark on an aged open enrichment row (including an older delayed mark), the actual enrichment adapter privacy-retirement callback preceding age skipping, existing historical parking preceding age skipping, and unknown non-positive epoch timestamps.

Final pre-push validation passed: 41 repository checks, backend typecheck (0 errors), selected unit hooks, and formatting.

Risk

Existing queued tasks continue to run; this only prevents new sweep re-wakes. Historical rows remain durable but will need a separately bounded backfill. The existing enrichment parking policy still applies to other eligible rows. No production access, merge, or deploy in this lane.

Rollback

Set BATCH_PIPELINE_SWEEP_AGE_GATE=off to restore existing sweeps. Only explicit off disables the new gate; malformed ages fall back to 14 days.

Failure-Class: FC-bounded-discovery-pinned-by-blocked-page