What is the Transactional Outbox pattern?

Transactional Outbox is an integration pattern that helps ensure reliable message delivery between microservices or systems, avoiding data consistency issues.

The idea is that when a business operation is performed in the database, a special table (outbox) is also updated to record events or messages that need to be sent. This entire operation occurs within a single transaction, guaranteeing atomicity.

A separate process or service periodically reads new messages from the outbox table and sends them to a messaging system (e.g., Kafka, RabbitMQ). After successful sending, the message is marked as processed.

Thus, reliable message delivery is achieved without the risk of losing events due to failures between writing to the database and sending the message.

An example of a simplified outbox table structure:

CREATE TABLE outbox (
  id SERIAL PRIMARY KEY,
  aggregate_id UUID,
  event_type VARCHAR(255),
  payload JSONB,
  processed BOOLEAN DEFAULT FALSE,
  created_at TIMESTAMP DEFAULT NOW()
);

In the code, during business logic execution within a transaction, a record is added to the outbox, and a separate worker reads and sends the events.