The Outbox Pattern: A SaaS Reliability Cheat Code

What you actually need to know

• The outbox pattern prevents split-state bugs caused by writing to a database and a message broker in separate operations.
• The pattern requires nothing special. One extra table in your existing database, a relay process, and idempotent consumers.
• The relay process is the seam that absorbs retries, backpressure, and delivery failures, so the business transaction does not have to.
• Downstream consumers must be idempotent. The pattern guarantees at-least-once delivery, not exactly-once.
• In my experience, teams that adopt the outbox pattern after their first dual-write incident adopt it permanently.

The core argument

Every SaaS that publishes events eventually runs into the same bug. The payment record is saved. The broker publish fails. The invoice email never sends. The customer waits. The support ticket arrives. The engineer adds a retry. The retry fires twice. The customer gets two emails. None of this is a clever failure mode. It is the predictable consequence of treating two side effects as a single operation when the runtime makes no such guarantee.

The outbox pattern is the fix. It does not prevent failures. It contains them. The idea is simple: write the business data and the event description into the same database transaction. If the transaction commits, the event exists durably. If it rolls back, the event was never created. There is no window where the database is updated and the event is not.

A relay process, running separately, reads undelivered events from the outbox table and sends them to their destination. If the send fails, the relay retries. If the relay crashes, it picks up where it left off on restart. The business transaction is already committed and safe. The delivery machinery can fail and recover without touching it.

The pattern is not exotic. It is a direct application of the rule that database writes are cheap and reliable, and network calls are expensive and unreliable. Put the state you need in the database first. Handle the network work afterwards, with retries.

How the implementation actually works

The outbox table

The table is small. An ID, the event type, the payload as JSON, the created timestamp, and a delivered flag. Some implementations add a processed-at timestamp and a retry count. That is all you need. The table lives in the same database as your business data. It is not a separate service.

``sql CREATE TABLE outbox_events ( id UUID PRIMARY KEY DEFAULT gen_random_uuid(), event_type TEXT NOT NULL, payload JSONB NOT NULL, created_at TIMESTAMPTZ NOT NULL DEFAULT now(), delivered_at TIMESTAMPTZ, retry_count INT NOT NULL DEFAULT 0 ); ``

The transaction

When you save a business record, you insert an outbox row in the same transaction. Both succeed or both fail. Nothing in between.

``sql BEGIN; INSERT INTO orders (id, customer_id, total) VALUES (...); INSERT INTO outbox_events (event_type, payload) VALUES ('order.created', '{"order_id": "..."}'); COMMIT; ``

The relay process

The relay polls the outbox table for undelivered rows. It sends each event to its destination, marks it delivered on success, or increments the retry count on failure. The relay runs as a background process, a cron job, or a Kubernetes sidecar. It is stateless. You can run multiple instances if you add a row-level lock on the poll query.

``sql SELECT * FROM outbox_events WHERE delivered_at IS NULL ORDER BY created_at ASC LIMIT 100 FOR UPDATE SKIP LOCKED; ``

FOR UPDATE SKIP LOCKED prevents two relay instances from grabbing the same row. This is the standard Postgres pattern for concurrent queue workers.

Change data capture as an alternative

For higher throughput, some teams use Postgres logical replication or Debezium to stream outbox rows to a broker instead of polling. The tradeoff is more infrastructure complexity in exchange for lower latency and no polling load on the database. For most SaaS teams under a few million events per day, polling is simpler and works fine.

What it actually requires

What to look for in a solid implementation

• Row-level locking on the relay poll so concurrent workers do not double-deliver.
• A dead-letter mechanism for events that exceed a retry threshold. Do not silently drop them.
• Monitoring on outbox queue depth. A growing queue means the relay is behind and you need to know.
• A purge job that removes delivered rows older than your retention window. The table should stay small.
• Payload versioning so older event shapes remain decodable after schema changes.
• Structured logging in the relay that includes event type, destination, retry count, and latency.

Expert opinion

The outbox pattern gets introduced after the first serious dual-write incident, almost every time. The reason teams do not adopt it earlier is that the failure mode is invisible until it is not. A message broker that drops a publish does not throw a visible exception in most configurations. The business transaction succeeds and the side effect silently vanishes. The outbox pattern makes the event visible before it is delivered, which is the correct place for durability to live.

>

Yashveer Singh, founder of Yashveer Labs

How this played out on a real project

A client running a booking platform had a recurring problem. Roughly one in eight hundred booking confirmations never resulted in a confirmation email. The database showed the booking created. The email service showed no inbound event. The broker logs showed a publish timeout that was being swallowed by a try-catch block that returned success to the caller. The booking was confirmed. The email was lost.

We added an outbox table to the bookings database in a single migration, wrapped the booking creation and outbox insert in one transaction, and built a relay process that polled every five seconds and delivered to the email service via an internal HTTP call. The silent failure rate dropped to zero in the first week. The relay has retried forty-three events in the months since, all of which eventually delivered.

The only ongoing cost is monitoring the outbox queue depth. We have an alert that fires if more than five hundred undelivered rows are older than two minutes. That alert has fired twice, both times because the email service was having an outage, not because the outbox was broken. In both cases the relay caught up automatically when the service recovered. For more on the async patterns that surround this one, see why your SaaS should have a job queue from day one and webhooks: the reliable pattern that most companies get wrong.

Common mistakes

1. Forgetting to make consumers idempotent. The outbox delivers at-least-once. A consumer that creates a duplicate record on a retry is not the outbox's fault, but the user experience is the same.
2. Not adding FOR UPDATE SKIP LOCKED to the relay query. Two relay instances will double-deliver without it.
3. Letting the outbox table grow without a purge policy. Delivered rows accumulate. The poll query gets slower. The relay falls behind.
4. Using a soft payload instead of the full event shape. If the consumer needs the order total, put the order total in the payload at write time. Do not make the consumer re-fetch it. Re-fetching introduces a new consistency window.
5. Swallowing relay errors without alerting. A silent relay is indistinguishable from a relay that is draining the queue. You need metrics.
6. Adding the outbox pattern to only some services. The benefit is consistency. If half your services still do direct broker publishes, half your events are still at risk.
7. Polling on too short an interval under high write volume. One second polling on a table receiving ten thousand inserts per minute will produce lock contention. Tune the interval to your actual latency requirements.

A 30-day plan

1. Week one. Identify the one service in your system most likely to produce a dual-write incident. This is usually the service that sends email or calls a webhook after a database write.
2. Week two. Add the outbox table. Write a migration. Wrap the two existing writes into a single transaction. Deploy and verify the queue fills and drains correctly.
3. Week three. Add observability. Queue depth metric, delivery latency histogram, dead-letter alert. Do not run this in production without at least one alert.
4. Week four. Review idempotency in every consumer that receives events from this service. Add deduplication where it is missing. Then expand the pattern to the next highest-risk service.

For the broader picture of how reliable messaging fits into a resilient backend, see the state machine pattern: a backend engineer's quiet hero and webhooks vs polling vs server-sent events vs WebSockets.