Have you had experience implementing 'exactly once' message delivery guarantees in Apache Kafka?

In Apache Kafka, a combination of several mechanisms is used to implement exactly-once delivery guarantees (exactly-once semantics, EOS):

• Using an idempotent producer, which assigns unique identifiers to messages, preventing duplication on retries.
• Kafka transactions, which allow grouping message sending and acknowledgment into an atomic operation.
• Consumers with transaction support, which read messages and acknowledge their processing within a transaction.

Example of using an idempotent producer in Java:

Properties props = new Properties();
props.put("bootstrap.servers", "localhost:9092");
props.put("enable.idempotence", "true");
props.put("key.serializer", "org.apache.kafka.common.serialization.StringSerializer");
props.put("value.serializer", "org.apache.kafka.common.serialization.StringSerializer");

KafkaProducer<String, String> producer = new KafkaProducer<>(props);

ProducerRecord<String, String> record = new ProducerRecord<>("topic", "key", "value");
producer.send(record);
producer.flush();
producer.close();

To fully ensure exactly-once, transactions should be used, and the producer and consumer should be configured correctly. This helps avoid message duplication and loss during failures.