Tell me about the Outbox Pattern — how does it work?

Outbox Pattern is a pattern for ensuring reliable message delivery between services in a distributed system, especially when it is necessary to guarantee consistency between the database and the messaging system.

How it works:

1. Instead of immediately sending an event to the message queue, the service records the event in a special "outbox" table within the same transaction where it modifies the main business data.
2. A separate process or worker periodically reads new entries from the outbox table and sends them to the messaging system (e.g., Kafka, RabbitMQ).
3. After successful sending, the event is marked as sent or removed from the table.

This achieves atomicity: either both the data and the event are saved together, or not saved at all, which prevents message loss or desynchronization.

An example of a simplified scheme:

// In a transaction
 tx.Exec("UPDATE orders SET status = ? WHERE id = ?", newStatus, orderId)
tx.Exec("INSERT INTO outbox (event_type, payload) VALUES (?, ?)", "OrderUpdated", payloadJson)
// commit tx

// In a separate worker
for {
  events := db.Query("SELECT * FROM outbox WHERE sent = false")
  for _, event := range events {
    sendToMessageBroker(event)
    markAsSent(event.id)
  }
  time.Sleep(pollInterval)
}