Ты говорил про идемпотентную отправку или Exactly-Once в Kafka. Чем из них пользовался?

Kafka supports idempotent message production, which guarantees that when the same message is sent multiple times, it will not be written to the topic more than once. This is implemented at the producer level using a unique producerId and sequence number for each message.

Exactly-Once Semantics (EOS) is a broader concept that ensures a message is processed exactly once, including transmission from the producer to the broker and further to the consumer, taking into account transactions.

In practice, I have used idempotent sending to ensure reliable message recording without duplicates. To implement EOS in Kafka, transactions need to be enabled, and transactional producers and consumers supporting transactions should be used.

Example of enabling idempotence in Java Kafka Producer:

Properties props = new Properties();
props.put("enable.idempotence", "true");
KafkaProducer<String, String> producer = new KafkaProducer<>(props);

For EOS, transactions are additionally configured:

props.put("transactional.id", "my-transactional-id");
producer.initTransactions();
producer.beginTransaction();
// message sending
producer.commitTransaction();

Thus, idempotence is part of the Exactly-Once mechanism, but EOS requires more complex configuration and support for transactions.