How to ensure message delivery guarantee in Kafka by the producer, even if the leader fails?

To guarantee message delivery in Kafka by the producer even in case of a leader failure, the following approaches should be used:

• Set acks=all — the producer will wait for acknowledgment from all replicas, ensuring that the message is written to all in-sync replicas (ISR).
• Use retries and error handling — in case of send errors (e.g., due to leader change), the producer should retry sending.
• Enable idempotence (enable.idempotence=true) — this prevents duplicate messages during retries.

An example of configuring a producer in Go using the sarama library:

config := sarama.NewConfig()
config.Producer.RequiredAcks = sarama.WaitForAll
config.Producer.Retry.Max = 5
config.Producer.Idempotent = true

producer, err := sarama.NewSyncProducer(brokers, config)
if err != nil {
    // handle error
}

msg := &sarama.ProducerMessage{
    Topic: "topic",
    Value: sarama.StringEncoder("message"),
}

partition, offset, err := producer.SendMessage(msg)
if err != nil {
    // handle error
}

fmt.Printf("Message is stored in partition %d, offset %d\n", partition, offset)

Thus, even if the leader changes, Kafka guarantees that the message will not be lost and will be delivered.