Are there issues with expanding (increasing) the number of partitions in Kafka?

In Kafka, you can increase the number of partitions for an existing topic, but it is not always straightforward and can lead to issues:

• Rebalancing keys: Increasing partitions changes the hashing of keys, which may cause messages with the same key to go to different partitions. This will break the message order by key.

• Consistency problems: Consumers, especially those using groups, may face rebalancing and temporary data loss.

• Increased load on the cluster: New partitions require resources, and if brokers are not prepared, it can affect performance.

• No backward reduction: After increasing partitions, you cannot decrease their number.

Therefore, increasing the number of partitions requires careful planning and understanding of the implications for message order and system load.