How is the idempotency of the consumer/producer implemented?

Idempotency of consumer and producer in messaging systems is achieved through the unique identification of messages and control of their processing.

For the producer, this can be generating a unique ID for each message (e.g., UUID). If the producer resends a message with the same ID, the broker or consumer can recognize the duplicate and ignore it.

For the consumer, idempotency is achieved by storing the state of processed messages (e.g., in a database or cache). When receiving a message with an already processed ID, the consumer does not perform reprocessing.

Example in Go for a producer generating UUIDs for messages:

import (
    "github.com/google/uuid"
)

func produceMessage(data string) Message {
    id := uuid.New().String()
    return Message{ID: id, Payload: data}
}

Thus, resending with the same ID can avoid duplication.

Some brokers (e.g., Kafka) also support idempotent sending at the protocol level, which simplifies implementation.