How does Kafka store messages and how does it provide them to consumers?

Kafka stores messages as sequential records in partitions of topics. Each topic is divided into several partitions, which are stored on broker disks as logs — ordered files where new messages are added to the end.

Consumers read messages by accessing a specific partition and offset — a unique message number within the partition. Kafka does not delete messages immediately after reading but retains them for a specified period or until the log size limit is reached, allowing consumers to read messages from a desired offset at any time.

Thus, Kafka implements a publish-subscribe model with message retention in logs and offset management on the consumer side.