What is a producer in Apache Kafka?

A producer in Apache Kafka is a client application that publishes messages to Kafka brokers.

Main functions:

1. Sending messages: Accepts data and sends it to a specified topic.
2. Serialization: Converts message data into bytes before sending.
3. Partitioning: Determines which partition of the topic the message will be sent to. By default, the message key's hash is used. If the key is absent, round-robin or sticky partitioning is used.
4. Buffering: Messages are temporarily accumulated in a buffer before being sent in batches, increasing throughput.
5. Error handling: Manages retries when temporary failures or broker errors occur.

Important configurations:

• bootstrap.servers: List of broker hosts and ports for initial connection.
• key.serializer: Class for serializing the message key.
• value.serializer: Class for serializing the message value.
• acks: Level of acknowledgment from brokers (0, 1, all).
• retries: Number of retry attempts for sending a message.

// Example of sending a message to Kafka using Java Producer API
Properties props = new Properties();
props.put("bootstrap.servers", "localhost:9092"); // Broker address
props.put("acks", "all"); // Require acknowledgment from all replicas
props.put("retries", 0); // No retries
props.put("linger.ms", 1); // Send messages immediately
props.put("key.serializer", "org.apache.kafka.common.serialization.StringSerializer"); // Key serializer
props.put("value.serializer", "org.apache.kafka.common.serialization.StringSerializer"); // Value serializer

Producer<String, String> producer = new KafkaProducer<>(props);
for (int i = 0; i < 100; i++) {
    producer.send(new ProducerRecord<String, String>("my-topic", Integer.toString(i), Integer.toString(i))); // Sending message
}

producer.close(); // Closing the producer