Senior
Kafka: 4 consumers in different consumer groups read from a single topic with 3 partitions — what will happen?
sobes.tech AI
Answer from AI
In Kafka, if we have a topic with 3 partitions and 4 consumers, each in its own separate consumer group, then each consumer will read all messages from all 3 partitions independently.
Explanation:
- Consumer group is a logical group of consumers that collectively read data from a topic, distributing partitions among themselves.
- If consumers are in different groups, each one receives a full copy of the data from the topic.
In our case:
- 4 consumer groups, one consumer in each.
- Each consumer reads all 3 partitions of the topic.
- Thus, each message from the topic will be read 4 times — once by each consumer from different groups.
If all 4 consumers were in one group, the partitions would be distributed among them (maximum 3 partitions, so one consumer would be left without data). But with different groups, each consumer receives the full stream.