What is the difference between Kafka and RabbitMQ?

Kafka and RabbitMQ are popular message brokers, but they have different architectural approaches and use cases.

• Kafka is a distributed data streaming platform focused on high throughput and message storage as logs. It is well-suited for processing large volumes of data, event streaming, and high-load systems. Kafka stores messages on disk and allows multiple consumers to read data independently.

• RabbitMQ is a classic message broker based on queues and message routing. It supports various protocols (AMQP, MQTT, etc.) and provides flexible routing, delivery acknowledgments, and complex messaging schemes. RabbitMQ is often used for microservice integration, guaranteed delivery tasks, and queue management.

Key differences:

Characteristic	Kafka	RabbitMQ
Model	Log of messages (topics, partitions)	Queues and exchanges (exchanges)
Message storage	Persistent, on disk	Usually deleted after delivery
Protocol support	Proprietary protocol	AMQP, MQTT, STOMP, etc.
Consumption approach	Consumers read by offset	Messages are deleted after processing
Use case scenarios	Analytics, streaming, big data	Integration, task processing, guaranteed delivery

Example of using Kafka in Go (producer):

package main

import (
    "github.com/segmentio/kafka-go"
    "context"
    "log"
)

func main() {
    writer := kafka.NewWriter(kafka.WriterConfig{
        Brokers: []string{"localhost:9092"},
        Topic:   "example-topic",
    })
    defer writer.Close()

    err := writer.WriteMessages(context.Background(),
        kafka.Message{
            Key:   []byte("Key-A"),
            Value: []byte("Hello Kafka"),
        },
    )
    if err != nil {
        log.Fatal("failed to write messages:", err)
    }
}

RabbitMQ in Go typically uses the library github.com/streadway/amqp for working with the AMQP protocol.