What rule exists for determining the ratio between the number of Consumers and Partitions in a Kafka system?

In Apache Kafka, there is a rule that the number of Consumers in a single Consumer Group cannot exceed the number of Partitions in the topic they are reading. This is because each Partition can only be processed by one Consumer from the group at a time. If there are more Consumers than Partitions, the extra Consumers will be idle without work.

Example:

• The topic has 4 Partitions
• There can be a maximum of 4 active Consumers in the Consumer Group

Therefore, to use resources efficiently, it is recommended to have the number of Consumers not exceeding the number of Partitions.