What was Kafka used for?

Apache Kafka is a distributed data streaming platform used for building high-throughput, low-latency message exchange systems.

Main applications of Kafka:

• Transmitting and processing large streams of data in real-time.
• Organizing message queues between microservices.
• Collecting and aggregating logs and metrics.
• Building event-driven architectures.

Kafka provides reliable message storage, scalability, and data processing capabilities through consumers.

Example of usage in Go with the sarama library:

package main

import (
    "github.com/Shopify/sarama"
    "log"
)

func main() {
    producer, err := sarama.NewSyncProducer([]string{"localhost:9092"}, nil)
    if err != nil {
        log.Fatal(err)
    }
    defer producer.Close()

    msg := &sarama.ProducerMessage{
        Topic: "test_topic",
        Value: sarama.StringEncoder("Hello Kafka"),
    }

    partition, offset, err := producer.SendMessage(msg)
    if err != nil {
        log.Fatal(err)
    }
    log.Printf("Message sent to partition %d at offset %d", partition, offset)
}