Can you explain how message distribution between partitions in Kafka is carried out and in what cases?

In Apache Kafka, messages are distributed among topic partitions to ensure scalability and parallel processing.

The distribution works as follows:

• If a key is specified when sending a message, Kafka uses a hash function of this key to determine which partition the message will go to. This guarantees that all messages with the same key will go to the same partition, maintaining order.
• If no key is specified, messages are distributed across partitions in a round-robin fashion or using other load balancing strategies, ensuring even load distribution.

Thus, the choice of partition depends on the presence of a key and the partitioning policy, which allows controlling message order and balancing load among partitions.

Example in Java using KafkaProducer:

ProducerRecord<String, String> record = new ProducerRecord<>("topic", "key1", "message");
producer.send(record);

Here, messages with the key "key1" will always go to the same partition.