How will the reading of topics and partitions be distributed among Processing Service instances (scaling)?

In message processing systems such as Kafka, topics are divided into partitions for parallel processing. When scaling the Processing Service, each consumer instance within a consumer group is assigned a specific set of partitions.

The distribution works as follows:

• All instances within a consumer group coordinate through a broker (e.g., Kafka).
• Each topic partition is assigned to exactly one consumer instance within the group.
• If the number of instances is less than the number of partitions, some instances will handle multiple partitions.
• If the number of instances exceeds the number of partitions, some instances will remain unassigned (idle).

Thus, scaling is achieved by increasing the number of instances that process different partitions of the topic in parallel. When instances are added or removed, rebalancing occurs, and partitions are redistributed among active instances.