Sobes.tech
Senior

If the last 5 messages out of a batch of 50 fail with an error during reading, what will happen to the offset?

sobes.tech AI

Answer from AI

If, when reading a batch of 50 messages, the last 5 fail with an error, then the offset behavior depends on the acknowledgment (commit) mechanism in the system used (for example, Kafka).

  • If the offset is committed after successfully processing the entire batch, then the offset will not be moved, and these 5 messages will be read again on the next read.
  • If acknowledgment occurs for each message, then the offset will only move to the last successfully processed message.

Thus, typically, the offset is not moved past messages with errors to ensure they are retried and to prevent data loss.