Senior
Was event loss in Kafka permissible? What delivery guarantees were used?
sobes.tech AI
Answer from AI
Event loss in Kafka is not allowed by default, but delivery guarantees depend on the settings of the producer, broker, and consumer.
The main delivery guarantees in Kafka:
- At most once — an event may be lost but is not duplicated.
- At least once — an event is delivered at least once, duplicates are possible.
- Exactly once — an event is delivered exactly once (using an idempotent producer and transactions).
To minimize losses, the following are used:
- Idempotent producer (enable.idempotence=true).
- Transactions for atomic sending of multiple messages.
- Topic replication with min.insync.replicas setting.
- Broker acknowledgments (acks=all).
Thus, event loss in Kafka is not the norm but depends on configuration and reliability requirements.