What is a partition in Kafka? How are partitions distributed among consumers of the same group?

A partition in Kafka is a logical part of a topic, representing an ordered sequence of messages. A topic is divided into multiple partitions for scalability and parallel processing.

Partition distribution among consumers in a group occurs so that each partition is assigned to exactly one consumer in the group. This ensures that each message is processed by only one consumer.

Partition allocation algorithms can vary (Range, RoundRobin, Sticky), but the goal is to evenly distribute the load and avoid duplicate processing.

For example, if a group has 3 consumers and the topic has 6 partitions, each consumer will receive approximately 2 partitions to process.