How is data stored in Kafka?

In Kafka, data is stored as sequences of messages (records) in topics, which are divided into partitions. Each partition is an ordered, immutable log of messages, where each message has a unique offset index.

Data is physically stored on Kafka broker disks as segment files. Messages are appended to the end of a partition, ensuring high write performance.

Main storage features:

• Messages are not deleted immediately but are stored for a specified retention period or until a size limit is reached.
• Consumers read messages by offset, allowing them to control which data has already been processed.

Thus, Kafka implements data storage as a distributed event log with scalability and high availability.