Kafka is unavailable, the service cannot send order updates. How to solve this problem? What is the Outbox pattern?

If Kafka is unavailable and the service cannot send a order update, it is necessary to ensure reliable message delivery and avoid data loss. One way is to use the Outbox pattern.

Outbox pattern is a template where state changes (for example, order updates) are first recorded in a local "outbox" table within the same transaction as the main data change. Then, a separate process or service reads these records from the table and sends them to Kafka. If Kafka is unavailable, the messages remain in the table and will be sent later when the service restores the connection.

Thus, reliable message delivery and data consistency are achieved.

An example of a simplified scheme:

1. In a transaction, update the order and record the event in the outbox table.
2. A separate worker periodically reads new records from the outbox and sends them to Kafka.
3. After successful sending, the event is marked as sent or removed from the outbox.

This prevents message loss during Kafka failures and guarantees that all updates will be delivered.

---

In Go, you can implement writing to the outbox within a database transaction, and then a separate goroutine sends events to Kafka.