How does Kafka achieve the absence of duplicates within a consumer group?

Kafka ensures the absence of duplicates within a consumer group through the mechanism of partition distribution of the topic among group consumers. Each partition is assigned to exactly one consumer in the group, which prevents parallel reading of the same partition by different consumers.

Additionally, Kafka uses offsets to track the reading position within a partition. Consumers periodically commit offsets, allowing them to resume reading from the last acknowledged position after a restart, thus avoiding reprocessing already read messages.

Therefore, the combination of unique partition assignment and offset management ensures that each message is processed exactly once within a consumer group, provided that the offset handling and committing are properly configured.