Sobes.tech
Senior

Have you used Kafka as storage? How many messages were stored in Kafka?

sobes.tech AI

Answer from AI

Kafka is typically used as a messaging system rather than a long-term data storage. However, thanks to the ability to store messages with a configured retention time, Kafka can temporarily hold data.

In real projects, Kafka can contain from a few thousand to millions of messages, depending on the configuration and load. For example, if it is a classified ads system, the number of ads in Kafka depends on the retention time and the volume of incoming data. Usually, Kafka stores data from a few hours to several days to ensure guaranteed delivery and the ability to read again.

An example of Kafka storage configuration:

log.retention.hours=168  # keep messages for 7 days
log.segment.bytes=1073741824  # segment size of 1 GB

Thus, Kafka can contain a large number of ads, but this does not replace a full-fledged storage, serving as a buffer and a transmission system.

Have you used Kafka as storage? How many messages… - sobes.tech