Senior
What happens if a service crashes and does not read messages from Kafka?
sobes.tech AI
Answer from AI
If a service that reads messages from Kafka crashes and does not read messages, then messages accumulate in Kafka topics. They are not deleted because Kafka retains data for a specified period or until a certain log size is reached.
As a result:
- The consumer does not offset (offset), so upon recovery, it will continue reading from the same place.
- If the service is down for a long time, the message queue grows, which can lead to increased processing delays.
- When disk space on Kafka brokers is exhausted, problems may occur, but usually Kafka is configured to delete old messages according to the retention policy.
Thus, a service crash leads to accumulation of unprocessed messages, but data is not lost, and after recovery, the service can continue processing from the last saved offset.