Sobes.tech
Middle+

What happens to a message in Kafka if an exception is thrown during its processing (but the application does not crash)?

sobes.tech AI

Answer from AI

If an exception is thrown while processing a message from Kafka in an application, but the application itself does not crash, the behavior depends on the processing method and consumer settings:

  • Automatic acknowledgment (auto-commit): if the offset is already committed, the message is considered processed and will not be redelivered, even if processing ended with an error.
  • Manual acknowledgment (manual commit): if the offset was not committed before the exception, the message will remain unprocessed and will be read again on the next run or restart of the consumer.

Thus, if the exception does not lead to committing the offset, the message will be redelivered and processed again. If the offset is already committed, the message will be lost in terms of reprocessing.

For reliable processing, manual acknowledgment and mechanisms for retries or DLQ (Dead Letter Queue) for messages that could not be processed are often used.