Sobes.tech
Senior

What happens if a service crashes and does not read messages from Kafka?

sobes.tech AI

Answer from AI

If a service that reads messages from Kafka crashes and does not read messages, then messages accumulate in Kafka topics. They are not deleted because Kafka retains data for a specified period or until a certain log size is reached.

As a result:

  • The consumer does not offset (offset), so upon recovery, it will continue reading from the same place.
  • If the service is down for a long time, the message queue grows, which can lead to increased processing delays.
  • When disk space on Kafka brokers is exhausted, problems may occur, but usually Kafka is configured to delete old messages according to the retention policy.

Thus, a service crash leads to accumulation of unprocessed messages, but data is not lost, and after recovery, the service can continue processing from the last saved offset.

What happens if a service crashes and does not read… - sobes.tech