Import Fathom recording media asynchronously

Merged
EH
2mo ago
+1497-178

Stack

  1. #25432 — Fathom media storage foundation
  2. #25433 — Asynchronous Fathom media import
  3. #25434 — Local recovery and disconnect cleanup

Supersedes closed #25337. All changes are inside the Fathom app; no Twenty server, core, or SDK changes.

Scope

Activate media imports from webhook, explicit sync, and backfill. Review against #25432.

  • One worker implements both stable request/import job entry points: claim, resolve owner, create or resume download, transfer, and settle.
  • Existing media-less records re-enqueue work, including retries after an enqueue failure. Preserve populated transcript/summary fields and guard concurrent upserts.
  • Conditional writes reject a replaced owner, claim, or download after provider/storage I/O. A matching upload checkpoint is resumed before consulting an expired provider URL.
  • Expired downloads and exhausted generation checks record explicit terminal media outcomes. Explicit Sync Fathom Call can retry them.
  • Preserve the existing persistent backfill schedule key AND stored value shape. Handle 429s with full Retry-After delays and a bounded retry budget.

Generation-status checks apply only to an already requested download. This does not poll workspace recordings or synchronize provider content edits. Scheduled local recovery is in #25434.

Review follow-up and limits

Fixed legacy records skipping media, lost enqueue retries, stranded terminal downloads, schedule compatibility, stale writes, stale contention re-enqueues, duplicated worker lifecycle, null checks, and missing pure error-mapping coverage.

Two limitations are explicit rather than hidden behind idempotency claims:

  • KV get/set pacing is best-effort, not atomic across concurrent workers. Retry-After handling limits the effect of collisions; no generic SDK locking primitive is added.
  • A provider create that succeeds before its ID can be persisted may be repeated on retry. There is no cross-system transaction or provider idempotency guarantee here.

Manifest default exports are required by the SDK extractor; handler exports remain named. Enqueue requests are chunked to the API's 200-item limit rather than rejecting valid multi-batch work.

Verification

Independently verified at this layer: typecheck, lint, 112 unit/real-HTTP tests across 21 files, and application build (28 files).

Added a real database integration regression for competing claims and stale writes after owner/download/claim replacement. It typechecks but was NOT run locally because Docker is unavailable; CI integration verification remains required. No service-mock suites or code comments added.

Review size: 997 non-test additions.