feat: switch to 1:1 eventType as a topic

Merged
HA
8mo ago
+838-4257

Summary by cubic

Switched to 1:1 EventType-as-topic names (no prefix) and moved DLQ handling fully in-process with per-event retry policies. Removed kafka-init, mapping code, and the generic dlq_events topic; fixed SSE OpenAPI media type and added ExecutionCancelled to both saga handling and SSE execution streams.

  • Refactors

    • Removed KafkaTopic and GroupId; producers publish to EventType string topics, consumers subscribe by EventType.
    • Deleted kafka/mappings.py and topic creation script; simplified topic logic.
    • DLQ: internal manager with preset per-event policies; removed DLQ routes, schemas, and E2E tests; DLQ events publish to their own EventType topics.
    • Replay configs now map EventType to string topic names; simplified service identifiers to string names.
    • Updated OpenAPI and regenerated frontend; SSE endpoints show text/event-stream; SSE execution stream includes ExecutionCancelled.
    • Moved non-E2E tests to unit; added unit app tests; consolidated subscribers into a single handler.

  • Migration

    • No manual topic creation; Kafka auto-creates topics (compose sets KAFKA_NUM_PARTITIONS=3).
    • Topic names match EventType strings with no prefix; update any dashboards/alerts expecting prefixed names.
    • DLQ REST endpoints are removed; retire any external integrations relying on them.
    • The generic dlq_events topic is removed; point any consumers to the DLQ EventType topics.

Summary by CodeRabbit

  • New Features

    • Per-event retry policies with several predefined strategies; topics are now derived from event types (string names).

  • Bug Fixes & Improvements

    • Simplified event routing and DLQ processing; services wait for Kafka health and default partitions set.

  • Removed

    • DLQ management HTTP endpoints, DLQ schemas and SDK methods, and related tests.

  • Docs

    • Architecture and deployment docs updated to reflect per-event-topic design and operational changes.