What types of message delivery guarantees are implemented in Kafka and how do they work in practice?

Kafka implements three main types of message delivery guarantees:

1. At most once

   • A message can be delivered 0 or 1 times.
   • No retries on failures, messages may be lost.
   • Used when speed is important and message loss is acceptable.

2. At least once

   • A message is guaranteed to be delivered, but may be delivered multiple times (duplicates).
   • Usually achieved through acknowledgments (acks) and retries on errors.
   • Requires duplicate handling on the consumer side.

3. Exactly once

   • A message is delivered and processed exactly once.
   • Implemented using Kafka transactions and idempotent producers.
   • Prevents duplicates and losses but requires additional configuration and resources.

In practice:

• For at least once, the producer usually waits for acknowledgment (acks=all), and the consumer commits offsets after processing.
• For exactly once, producer idempotence (enable.idempotence=true) is enabled, and transactions are used.

This allows balancing between performance and message delivery reliability.