Kafka delivery semantics: exactly once, at least once, at most once. Does Kafka support these semantics from producer to consumer?
sobes.tech AI
Answer from AI
Kafka has three main message delivery semantics:
- At most once — a message is delivered no more than once, but message loss is possible.
- At least once — a message is delivered at least once, which may result in duplicates.
- Exactly once — a message is delivered exactly once without loss or duplicates.
Kafka initially supports the at least once semantics. This means messages can be delivered multiple times, for example, if a consumer does not acknowledge receipt.
To implement exactly once semantics, Kafka provides support for transactions at the producer and consumer level starting from version 0.11.0.0. This guarantees that messages are written and processed exactly once when using transactions and idempotent producers.
The at most once semantics is achieved if the producer sends messages without acknowledgments, but this is not typical for Kafka.
Thus, Kafka supports all three semantics, but exactly once requires proper configuration and the use of transactions and idempotent producers.