Explain the mechanism of consumer interaction with topics in Apache Kafka

In Apache Kafka, a consumer interacts with topics as follows:

• The consumer subscribes to one or more topics.
• A topic is divided into partitions — units of parallelism.
• The consumer reads messages from partitions, starting from a specific offset.
• Kafka guarantees that messages within a partition are ordered, and the consumer reads them sequentially.
• Consumers can be grouped into consumer groups to distribute the load: each partition is processed by only one consumer in the group.
• The consumer periodically commits the offset to record which messages have been processed.

Example in Java using KafkaConsumer:

Properties props = new Properties();
props.put("bootstrap.servers", "localhost:9092");
props.put("group.id", "my-group");
props.put("key.deserializer", "org.apache.kafka.common.serialization.StringDeserializer");
props.put("value.deserializer", "org.apache.kafka.common.serialization.StringDeserializer");

KafkaConsumer<String, String> consumer = new KafkaConsumer<>(props);
consumer.subscribe(Arrays.asList("my-topic"));

while (true) {
    ConsumerRecords<String, String> records = consumer.poll(Duration.ofMillis(100));
    for (ConsumerRecord<String, String> record : records) {
        System.out.printf("offset = %d, key = %s, value = %s%n", record.offset(), record.key(), record.value());
    }
    consumer.commitSync();
}

Thus, the consumer reads messages from the topic, processes them, and records the reading progress.