Sobes.tech
Senior

Kafka: how is reading distributed with a topic with 3 partitions and 2 consumers in one group?

sobes.tech AI

Answer from AI

In Kafka, topic partitions are distributed among consumers within a single group so that each partition is read by exactly one consumer from that group. For a topic with 3 partitions and 2 consumers in the group, the distribution will be approximately as follows:

  • One consumer will receive 2 partitions
  • The other consumer will receive 1 partition

This is because the number of partitions exceeds the number of consumers, and Kafka tries to distribute partitions as evenly as possible. If there were more or an equal number of consumers to partitions, some consumers might not receive any data.

Example of distribution:

Consumer Partitions
Consumer 1 0, 1
Consumer 2 2

Thus, reading will be parallel, but each partition will be read by only one consumer, ensuring message order within each partition.