Junior — Middle
How does Kafka handle messages that were not successfully processed?
sobes.tech AI
Answer from AI
In Kafka, messages are written to topics and consumed by subscribers. If a message was not successfully processed by the consumer, Kafka itself does not delete the message from the topic, but leaves it available for re-reading.
Handling of unsuccessful messages is usually implemented on the consumer side:
- The consumer may not commit the offset of the message, which will lead to re-reading this message on the next run.
- Retry mechanisms can be used, or problematic messages can be sent to a separate "dead-letter" topic for further analysis.
Thus, Kafka provides reliable message storage, while error handling and retry logic are implemented in the consumer application.