Sobes.tech
Senior

Kafka: 4 consumers in different consumer groups read from a single topic with 3 partitions — what will happen?

sobes.tech AI

Answer from AI

In Kafka, if we have a topic with 3 partitions and 4 consumers, each in its own separate consumer group, then each consumer will read all messages from all 3 partitions independently.

Explanation:

  • Consumer group is a logical group of consumers that collectively read data from a topic, distributing partitions among themselves.
  • If consumers are in different groups, each one receives a full copy of the data from the topic.

In our case:

  • 4 consumer groups, one consumer in each.
  • Each consumer reads all 3 partitions of the topic.
  • Thus, each message from the topic will be read 4 times — once by each consumer from different groups.

If all 4 consumers were in one group, the partitions would be distributed among them (maximum 3 partitions, so one consumer would be left without data). But with different groups, each consumer receives the full stream.