Sobes.tech
Senior

In a topic with 6 partitions, 6 more were added. The key is — userID. Will a message with the same key go to the same partition?

sobes.tech AI

Answer from AI

In Kafka, a partition is determined by a hash function of the message key modulo the number of partitions. If a topic initially had 6 partitions and then 6 more were added (total 12), changing the number of partitions will alter the hashing.

This means that a message with the same key, such as userID, is likely to be assigned to a different partition because the number of partitions affects the calculation of the partition index:

partition = hash(key) % num_partitions

As a result, when the number of partitions increases, messages with the same key may be distributed across different partitions, which can affect the order of message processing with that key.

To maintain consistent key distribution, topics are usually created with a fixed number of partitions or special partitioning strategies are used.

In a topic with 6 partitions, 6 more were added. The… - sobes.tech