Sobes.tech
Senior

How are Topic and Partition related in Kafka?

sobes.tech AI

Answer from AI

In Apache Kafka, a Topic is a logical category or stream of messages to which data is published. Each Topic is divided into several Partitions — physical parts that store messages in an ordered manner.

The relationship between them:

  • A Topic consists of one or more Partitions.
  • A Partition is a sequence of messages stored on Kafka brokers.
  • Dividing a Topic into Partitions allows for scaling data processing and provides parallelism: different consumers can read different Partitions simultaneously.
  • The order of messages is guaranteed within a single Partition, but not across Partitions.

Thus, a Partition is a unit of parallelism and scaling within a Topic, and a Topic is a logical group of messages that aggregates these Partitions.