What tasks did you perform with Kafka? How did you configure delivery guarantees (exactly-once)?

With Kafka, I usually solved tasks related to organizing reliable and scalable message transmission between microservices, real-time event processing, and building high-throughput systems.

To configure exactly-once delivery guarantees, I used the following approaches:

• Enabled transaction support in Kafka Producer to ensure atomic message sending.
• Used an idempotent producer (enable.idempotence=true) to avoid message duplication.
• On the consumer side, applied offset management within transactions to guarantee that each message is processed exactly once.

An example of configuring a producer in Go using the sarama library:

config := sarama.NewConfig()
config.Producer.Idempotent = true
config.Producer.RequiredAcks = sarama.WaitForAll
config.Producer.Transaction.ID = "my-transactional-id"

producer, err := sarama.NewSyncProducer(brokers, config)
if err != nil {
    panic(err)
}

// Begin transaction
producer.BeginTxn()

// Send messages
producer.SendMessage(&sarama.ProducerMessage{Topic: "topic", Value: sarama.StringEncoder("message")})

// Commit transaction
producer.CommitTxn()

This way, exactly-once semantics are achieved when working with Kafka.