How does the producer decide which partition to send data for writing?

A producer in partitioned systems (e.g., Kafka) decides which partition to send data to by using the message key and a partitioning function.

Main methods of partition selection:

1. By message key:

   • If the message has a key, the producer applies a hash function to the key.
   • The hash result is taken modulo the number of partitions.
   • This guarantees that all messages with the same key go to the same partition, maintaining order.

2. Without a key:

   • The producer can distribute messages across partitions cyclically (round-robin) for load balancing.

Example in Go (simplified):

func choosePartition(key string, partitions int) int {
    if key == "" {
        // round-robin or random selection
        return rand.Intn(partitions)
    }
    hash := fnv.New32a()
    hash.Write([]byte(key))
    return int(hash.Sum32()) % partitions
}

Thus, the producer controls message routing, ensuring load balancing and order where necessary.