Have there been situations where messages in Kafka did not arrive?

Yes, situations where messages in Kafka are not delivered are possible. The main reasons include:

• Network failures between the producer and broker, causing message delivery failure.
• Configuration errors (e.g., incorrect topic, insufficient permissions).
• Insufficient replication or broker failures leading to data loss.
• Incorrect use of acknowledgments (acks): if the producer does not wait for broker confirmation, the message may be lost.

To minimize losses, it is recommended to:

• Use acks=all to confirm writes to all replicas.
• Enable retries for sending messages.
• Monitor broker and topic status.
• Handle send errors in producer code.

Example of sending a message with confirmation in Go:

producer, _ := sarama.NewSyncProducer(brokers, config)
msg := &sarama.ProducerMessage{Topic: "topic", Value: sarama.StringEncoder("message")}
partition, offset, err := producer.SendMessage(msg)
if err != nil {
    // handle send error
}

Thus, proper configuration and error handling help reduce the likelihood of message loss.