updates: faststream, start/stop (almost) removed

Merged
HA
9mo ago
+1399-3213

Summary by cubic

Switched event handling to FastStream KafkaBroker and removed nearly all manual start/stop lifecycle code. Events are now consumed via FastStream subscribers with Avro decoding, and services publish through a broker-backed UnifiedProducer.

  • New Features

    • Added FastStream KafkaBroker with Dishka integration for FastAPI and worker processes.
    • Introduced an Avro decoder using SchemaRegistryManager for subscriber payloads.
    • Registered FastStream subscribers for EventStore, notifications, SSE routing, coordinator, result processor, k8s worker, pod monitor, saga orchestrator, and DLQ processor.
    • UnifiedProducer now uses KafkaBroker; DLQManager publishes via the broker.

  • Refactors

    • Removed UnifiedConsumer, EventDispatcher, DLQ handler helpers, consumer types, and idempotency middleware.
    • Simplified DI providers and containers to accept KafkaBroker; lifecycle is handled by FastStream instead of manual start/stop.
    • Updated app startup to initialize the broker and wire Dishka for both FastAPI and FastStream.
    • Reworked workers to create the broker and register subscribers directly; simplified PodMonitor and tests accordingly.
    • Dependencies: added faststream[kafka], upgraded dishka to 1.7.2, pinned aiokafka to 0.12.0 for remaining test usage.

Summary by CodeRabbit

  • New Features

    • Central Kafka broker with Avro decoding and managed lifecycle; workers now run via FastStream lifecycles and register event subscribers.

  • Bug Fixes

    • More reliable DLQ handling with improved retry policies, filtering, and guaranteed broker shutdown.

  • Refactor

    • Simplified event pipeline and lifecycles: moved to broker + scheduler model; removed legacy consumer/dispatcher/idempotency indirection.

  • Chores

    • Dependency updates and test adjustments.