Junior — Middle
How to guarantee message sequence preservation when sending data to Kafka if no key is used?
sobes.tech AI
Answer from AI
In Apache Kafka, message order is guaranteed within a single partition. If no key is used when sending messages, by default, messages are distributed randomly across partitions, and order between different partitions is not guaranteed.
To ensure the preservation of message sequence without a key, you can:
- Send all messages to the same partition, explicitly specifying the partition during sending.
- Use a key that is the same for all messages so they go to the same partition.
Example of sending with partition specification in Python (kafka-python):
from kafka import KafkaProducer
producer = KafkaProducer(bootstrap_servers='localhost:9092')
# Send message to partition 0
producer.send('topic_name', value=b'message', partition=0)
producer.flush()
Thus, the sequence is preserved because all messages go to the same partition.