Как обеспечить exactly-once семантику в Kafka?

Achieving exactly-once semantics in Kafka is done through transactions and an idempotent producer. Key points:

• Enable idempotence on the producer (enable.idempotence=true) to avoid duplicate messages on retries.
• Use transactions (initTransactions(), beginTransaction(), commitTransaction()) to ensure atomicity of multiple message writes.
• Consumers should use read_committed mode to read only committed transactions.

Example of initializing a transactional producer in Java:

Properties props = new Properties();
props.put(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG, "localhost:9092");
props.put(ProducerConfig.ENABLE_IDEMPOTENCE_CONFIG, "true");
props.put(ProducerConfig.TRANSACTIONAL_ID_CONFIG, "my-transactional-id");

KafkaProducer<String, String> producer = new KafkaProducer<>(props, new StringSerializer(), new StringSerializer());
producer.initTransactions();

try {
    producer.beginTransaction();
    producer.send(new ProducerRecord<>("topic", "key", "value"));
    // other operations
    producer.commitTransaction();
} catch (Exception e) {
    producer.abortTransaction();
}

This ensures exactly-once delivery and processing of messages in a distributed system.