How are messages distributed among partitions in Kafka with 6 partitions and 6 consumers? What if there are 2 consumers?

In Kafka, messages are distributed across topic partitions. With 6 partitions and 6 consumers in the same group, each consumer usually receives exactly one partition, meaning the load is evenly distributed.

If the number of consumers decreases to 2, then the 6 partitions will be distributed between these two consumers. For example, one might get 3 partitions, and the other — 3. It is important that the number of partitions is always greater than or equal to the number of consumers; otherwise, some consumers will be idle.

Distribution example:

• 6 partitions, 6 consumers: each consumer reads 1 partition.
• 6 partitions, 2 consumers: consumer 1 reads partitions 0,1,2; consumer 2 — 3,4,5.

This distribution ensures parallel data processing and load balancing among consumers.