Sobes.tech
Senior

What is the disadvantage of using material code as a partition key in Kafka?

sobes.tech AI

Answer from AI

Using material code as a partition key in Kafka has the following drawback:

If the material code has low cardinality or is unevenly distributed (for example, some codes occur much more frequently), then partitions will be unevenly distributed. This will lead to:

  • Hot partitions: Some partitions will be overloaded, which will degrade performance and increase latency.

  • Loss of load balancing: Kafka will not be able to efficiently utilize all partitions and consumers.

To avoid this, it is recommended to use keys with good distribution or apply additional hashing/mixing of keys for even message distribution across partitions.