Multiple consumer instances read from Kafka — what are the limitations?

When reading from Kafka with multiple consumer instances, there are important limitations and features:

• Consumer Groups: Each instance must be in one group so that Kafka distributes partitions among them. Then each partition is read by only one instance.
• Limit on the number of instances: The number of active consumers in a group should not exceed the number of partitions in the topic, otherwise some instances will be idle.
• Offset management: Properly manage offsets to avoid re-reading or skipping messages.
• Load balancing: When adding or removing instances, rebalancing occurs, which can cause temporary unavailability.

Example: if a topic has 3 partitions and you have 5 consumer instances in one group, only 3 of them will receive data, the rest will be idle.