How does Kafka distribute messages across partitions?

Kafka distributes messages across topic partitions using partitioning to ensure scalability and order.

Main distribution methods:

1. Message key: If a message has a key, Kafka uses a hash function (usually murmur2) on the key to determine which partition to send the message to. This guarantees that all messages with the same key will go to the same partition, maintaining order.

2. Manual partition assignment: The producer can explicitly specify the partition number when sending.

3. Round-robin: If there is no key and no partition specified, the producer distributes messages across partitions cyclically to evenly load the brokers.

Thus, keys provide logical grouping and order, while absence of a key balances the load.

Example in Java:

ProducerRecord<String, String> record = new ProducerRecord<>("topic", key, value);
producer.send(record);

Kafka will automatically compute the partition based on the key.