Tell about the types of delivery guarantees in Kafka, how they differ, and when to use which.

Kafka has three main levels of message delivery guarantees:

1. At most once

   • A message may be delivered 0 or 1 times.
   • Used when message loss is acceptable, but duplication is not.
   • Usually achieved without acknowledgments from the broker.

2. At least once

   • A message is guaranteed to be delivered, but may be delivered multiple times (duplication possible).
   • This is the standard Kafka mode with acknowledgments (acks=all) and retries.
   • Suitable for systems where data loss is unacceptable, but duplication can be handled.

3. Exactly once

   • Guarantees that each message will be processed exactly once.
   • Implemented using Kafka transactions and idempotent producers.
   • Used in critical systems where data accuracy is important.

The choice depends on system requirements:

• If speed is important and message loss is acceptable — at most once.
• If reliability is important and duplication can be handled — at least once.
• If strict accuracy is needed — exactly once, but with additional overhead.