How to write a test for Kafka (producer/consumer)?

For testing Kafka producer and consumer in Java, the spring-kafka-test library or EmbeddedKafka from Spring Kafka is often used, which allows running an embedded Kafka broker in memory.

Example of a simple test using EmbeddedKafka:

@SpringBootTest
@EmbeddedKafka(partitions = 1, topics = {"test-topic"})
public class KafkaIntegrationTest {

    @Autowired
    private KafkaTemplate<String, String> kafkaTemplate;

    @Autowired
    private ConsumerFactory<String, String> consumerFactory;

    @Test
    public void testSendReceive() throws Exception {
        // Send a message
        kafkaTemplate.send("test-topic", "key1", "Hello Kafka");

        // Create a consumer for verification
        Consumer<String, String> consumer = consumerFactory.createConsumer();
        consumer.subscribe(Collections.singleton("test-topic"));

        ConsumerRecords<String, String> records = KafkaTestUtils.getRecords(consumer);

        assertThat(records.count()).isGreaterThan(0);
        ConsumerRecord<String, String> record = records.iterator().next();
        assertEquals("Hello Kafka", record.value());

        consumer.close();
    }
}

This approach allows verifying that the producer correctly sends messages and the consumer receives them. For unit tests, you can mock KafkaTemplate or Consumer, and for integration tests, use EmbeddedKafka.