Does Kafka guarantee message delivery? Can Kafka lose a message if the producer does not receive an error?

Kafka provides message delivery guarantees, but with certain conditions and configurations.

By default, Kafka supports three levels of delivery:

• At most once — messages may be lost if the producer does not receive acknowledgment.
• At least once — messages are not lost but may be duplicated.
• Exactly once — a complex configuration that ensures delivery without losses and duplicates.

If the producer does not receive an error, it means the message was accepted by the broker and logged, so loss is unlikely. However, if the producer does not receive acknowledgment due to network issues, the message may be sent again, leading to duplicates.

Example of producer configuration in Go with acknowledgments:

producer, err := sarama.NewSyncProducer(brokers, &sarama.Config{
    Producer: sarama.ProducerConfig{
        RequiredAcks: sarama.WaitForAll, // wait for acknowledgment from all replicas
        Retry: sarama.ProducerRetry{
            Max: 5,
        },
    },
})

Thus, Kafka does not lose messages with proper configuration and error handling, but it is important to correctly handle acknowledgments and errors on the producer side.