Junior — Middle
In what situations can data loss occur in Kafka?
sobes.tech AI
Answer from AI
Data loss in Kafka can occur in the following situations:
- Retention setting (storage time) is too short. If data is deleted from the topic before the consumer reads it.
- Incorrect acknowledgment (acks) settings. For example, with acks=0, the producer does not wait for acknowledgment, which can lead to message loss.
- Errors in consumer processing. If the consumer does not save offsets or does so incorrectly, data may be skipped.
- Hardware failures without replication. If the topic has no replicas and the broker crashes, data can be lost.
- Errors during message production. For example, exceeding message size or network issues.
To minimize losses, it is recommended to use replication, correct acks settings (e.g., all), monitor retention, and properly manage consumer offsets.