Where inside Kafka is the Offset value for consumed messages stored?

In Apache Kafka, the Offset value for consumed messages is stored in a special internal topic called __consumer_offsets. This topic holds information about the current position of each consumer group in each partition of the topic.

Thus, Kafka does not store the offset locally on the client but centrally in the cluster, allowing consumers to recover and continue reading from the correct position.

Additionally, clients can manually commit offsets or do so automatically, depending on the consumer settings.

Example in Go using the confluent-kafka-go library for committing offsets:

consumer.CommitMessage(message)

where message is the read message, and its offset will be saved.