[deploy] Migrate Redpanda to Kafka

Merged
AS
5d ago
+5730-116

Phase1 Kafka-only production cutover is COMPLETE under the user-approved Q4 packet. Kafka4.4.0 is live with remote storage disabled; the approved30-minute observation passed. Existing app/Connect image contracts and the repaired history image were preserved, except the explicitly approved narrow worker update. No repeat cutover is requested.

  • Execution and sanitized observation receipt: same six topics, three4GiB sources/three1GiB quarantines,128MiB segments, seven-day ceilings and20GiB reserve. These are selected phase1 policies, not hard disk caps or guaranteed replay hours.
  • Shared application release proposal: merging can trigger the normal full app/Cloudflare TF release and matching mobile workflows. User Q5/A (“Approved.”) now authorizes this broader normal production release.
  • Review image receipt: source3350, digest sha256:54214d6276f03c5d96ae02ba75e91b7fc0fecd63c8b416ec36b7548a866a651b; packaged retention-phase and current-history-source checks passed. Later769 changes deploy rendering, workflow fixtures, fresh-host bootstrap/docs; app executable paths are unchanged. The normal release must resolve its own exact source/image.
  • Review fixes include pre-secret activation/storage/API/topic gates, literal topic names, Kafka convergence checks, safe structural cluster-ID env rendering, complete capture assertions and canonical fresh-host data ownership.69 focused deployment/renderer tests pass locally; Current-head CI42556341745 passed at7699423c9:95successful jobs,0failures; all9required branch-protection checks pass.
  • Read-only Cloudflare plan42549866789 reports No changes. Already-managed R2 resources remain declared. The separate OCI cloud-init correction is offline validated and NOT applied/planned; future OCI apply requires replacement review.

User Q5/A (“Approved.”) explicitly accepted the Terraform-only GitHub managing-credential exception and the existing unauthenticated internal-overlay broker trust boundary, and authorized merging #3233 with the documented normal automatic release. Both policy threads are resolved with that decision. No credential values or configuration sources are changed. The earlier readiness preparation itself made no further production changes.

Dependent PR #3236 owns stock Aiven/R2 tiering and remote retention verification. It is the SECOND release on the SAME Kafka data/cluster/topic identities. Retained old data, explicit erasure-scrub combination and phase2 capacity/backend/release gates remain; no unrelated erasure redesign, new schema migration or ClickHouse upgrade is included.