What guarantees of delivery exist in Kafka? Tell me about At Most Once, At Least Once, Exactly Once.

Kafka has three main message delivery guarantee models:

1. At Most Once

   • A message can be delivered 0 or 1 times.
   • No retries on errors, so message loss is possible.
   • Used when speed is important and message loss is acceptable.

2. At Least Once

   • A message will be delivered at least once.
   • Duplicates may occur if a failure happens during acknowledgment and the message is resent.
   • This is the standard mode of operation in Kafka when using acknowledgments (acks) and retries.

3. Exactly Once

   • Guarantees that each message is processed exactly once.
   • Achieved in Kafka through idempotent producers and transactions.
   • Prevents duplicates and loss but requires additional configuration and resources.

Example usage:

// Example producer configuration for idempotent sending (Exactly Once)
producer, err := kafka.NewProducer(&kafka.ConfigMap{
    "bootstrap.servers": "localhost:9092",
    "enable.idempotence": true, // enables idempotence
    "acks": "all",
})

Thus, the choice of delivery guarantee depends on the system's reliability and performance requirements.