What retention settings are available in Kafka for data storage?

In Kafka, the following main parameters are used to configure data retention:

• retention.ms — the time messages are stored in milliseconds. After this time, messages are deleted.
• retention.bytes — the maximum size of a log segment in bytes. If the log size exceeds this value, old data is deleted.
• log.retention.check.interval.ms — the interval for checking the log for the need to delete outdated data.

These parameters can be configured at the topic or broker level. For example, to store data for no more than 7 days, set retention.ms=604800000.

There is also an option to configure the deletion policy:

• delete — delete outdated messages (default)
• compact — keep only the latest messages with a unique key (log compaction)

These settings allow balancing between data retention time and disk space usage.