Is it possible to completely delete messages from Kafka after they are published?

In Apache Kafka, messages are not deleted immediately after publication because Kafka is designed as a distributed event log with the ability to store message history.

Message deletion occurs based on the following principles:

• Messages are automatically deleted according to storage settings (retention policy), for example, by time (retention.ms) or by topic size (retention.bytes).
• It is possible to configure message deletion using the policy of compacted topics, where only the latest values by key are retained.

It is not possible to completely delete a specific message immediately after publication, as Kafka does not support message deletion by ID. You can only set a retention period after which messages will be deleted.

If you need to "delete" a message, usually the following are used:

• Setting a short retention time for the topic.
• Sending special messages with a null value (tombstone) for compacted topics to mark the record as deleted.

Thus, complete immediate deletion of messages in Kafka is impossible, but you can manage their retention time and cleanup policies.