Senior
What is the disadvantage of using material code as a partition key in Kafka?
sobes.tech AI
Answer from AI
Using material code as a partition key in Kafka has the following drawback:
If the material code has low cardinality or is unevenly distributed (for example, some codes occur much more frequently), then partitions will be unevenly distributed. This will lead to:
-
Hot partitions: Some partitions will be overloaded, which will degrade performance and increase latency.
-
Loss of load balancing: Kafka will not be able to efficiently utilize all partitions and consumers.
To avoid this, it is recommended to use keys with good distribution or apply additional hashing/mixing of keys for even message distribution across partitions.