What delivery guarantee mechanism was used in Kafka at your previous workplace?

In Kafka, message delivery guarantees are ensured through producer and consumer settings, as well as acknowledgment (acks) mechanisms and offset management.

Main mechanisms:

• acks — a producer parameter that determines how many replicas must confirm the message write. Values:

  • 0 — no acknowledgment wait (low reliability)
  • 1 — wait for acknowledgment from the leader
  • all (or -1) — wait for acknowledgment from all in-sync replicas (ISR), ensuring maximum reliability

• Retry and idempotence — the producer can resend messages on errors, and idempotence prevents duplication.

• Offset commit — the consumer commits offsets after processing messages to ensure messages are not processed multiple times.

In practice, for "at least once" delivery guarantees, acks=all and an idempotent producer are used, while for "exactly once" delivery, additional configurations and application-level handling are required.

Example of producer configuration in Go (using sarama):

config := sarama.NewConfig()
config.Producer.RequiredAcks = sarama.WaitForAll
config.Producer.Idempotent = true
config.Producer.Retry.Max = 5
producer, err := sarama.NewSyncProducer(brokers, config)

Thus, the delivery guarantee mechanism is a combination of acknowledgments, replication, and offset management.