Middle
Have there been cases where an event did not arrive or duplicates occurred? What was done about it?
sobes.tech AI
Answer from AI
Yes, in real systems, there are often cases where events do not reach the consumer or arrive as duplicates. This is due to network failures, processing errors, message re-sending, etc.
Several approaches are used to address these issues:
- Idempotency — processing the event so that receiving it multiple times does not affect the result (for example, checking for uniqueness by ID).
- Acknowledgements — the consumer confirms receipt of the event, and the sender resends if no confirmation is received.
- Using delivery-guaranteed queues — for example, RabbitMQ, Kafka, which ensure reliable delivery and allow duplicate control.
- Logging and monitoring — to identify and analyze missed or duplicated events.
An example in Python using Redis Stream, where processed messages are tracked to avoid duplicates:
processed_ids = set()
for message in redis_stream.read():
msg_id = message['id']
if msg_id in processed_ids:
continue # skip duplicate
process(message)
processed_ids.add(msg_id)
Thus, a combination of reliable delivery mechanisms and idempotent processing helps minimize issues with event loss and duplicates.