Read Replicas: When They Save You and When They Lie to You
A read replica is a copy of the primary database that receives a continuous stream of write operations from the primary and applies them to stay in sync. Read replicas are used to distribute read traffic: queries that do not need the absolute latest data are routed to replicas, reducing load on the primary. The critical constraint is replication lag: replicas are always slightly behind the primary (milliseconds to seconds depending on load), so reads from replicas may return data that is slightly stale.
Written by Yashveer Singh, founder of Yashveer Labs.
What you need to know
- Read replicas solve read load scalability problems. They are not the right solution for write load problems, connection limit problems, or query performance problems.
- Replication lag is always nonzero. Any code path that writes and then reads expecting the write to be visible must account for lag.
- The read your own writes problem is the most common bug introduced by read replica routing. Test this specific scenario explicitly.
- Read replicas provide availability benefits: if the primary fails, a replica can be promoted. This is separate from the read load distribution benefit.
- Route analytics, reporting, and background job queries to replicas. Keep user-facing transactional reads that follow writes on the primary.
The core argument
Read replicas are presented as a straightforward scaling solution: add a replica, route reads to it, halve the load on the primary. In practice, the routing decision is more nuanced, and teams that route all reads to replicas without thinking through the consistency implications introduce bugs that are difficult to reproduce because they depend on replication timing.
The right model for thinking about read replica routing is: separate reads by whether they need to see data from the current user session's most recent writes. Analytics queries that aggregate across all users do not need to see writes from the last 100 milliseconds. A user profile page shown immediately after the user updated their profile does need to see the write. The former is safe for a replica. The latter must go to the primary.
The architectural pattern I find most reliable is: writes always go to the primary, reads that immediately follow writes in the same request or user session go to the primary, everything else goes to the replica. This routing can be expressed in the data access layer: a request context that tracks whether any writes have occurred in the session, and a database client that routes based on that flag. On an e-commerce project, this pattern eliminated the stale product inventory bugs we were seeing when a purchase was completed and the confirmation page showed the pre-purchase stock level from a lagged replica.
Common mistakes
- Routing all reads to replicas without considering the write-then-read pattern. The most common manifestation: a user submits a form, is redirected to a success page, and the success page shows the old data because the replica has not caught up. The fix is routing the success page's read to the primary, or adding a short delay or retry before reading from the replica.
- Not monitoring replication lag. Replication lag that is normally negligible can spike under high write load. Without monitoring, a sudden lag increase is invisible until users start reporting stale data. Monitor replica lag as a standard infrastructure metric and alert on sustained lag above a threshold.
- Using read replicas to fix query performance problems. If a query is slow, moving it to a replica makes it slow on the replica instead of the primary. The performance problem (missing index, inefficient query plan) still exists. Fix the query first. Use replicas for distributing load, not for avoiding query optimization.
- Not testing replica failover. If the primary fails and a replica is promoted, the new primary starts accepting writes. Untested failover paths may have configuration issues (replica not configured for writes, application connection strings not updated) that are only discovered during an actual failure. Test failover in a staging environment before relying on it.
- Opening too many connections to the replica. The connection limit problem that read replicas are sometimes used to solve (the primary has too many connections) reappears on the replica. Read replicas have the same connection limits as the primary. PgBouncer connection pooling should be deployed in front of replicas just as it should be in front of the primary.
Where to start
- Identify the actual load pattern before adding replicas. Check whether the primary's bottleneck is read load (high query count, high CPU from read queries) or something else (write load, connection limits, missing indexes). Read replicas only help with read load. A slow query profile on the primary shows whether optimization would reduce load more than replication.
- Start by routing only analytics and reporting queries to the replica. These queries are write-decoupled by nature, high-impact on primary load, and safe if the replica is slightly stale. Add the replica client, route reporting queries to it, and monitor the primary load reduction and replica lag. This validates the replica setup without risking read-your-own-writes bugs in user-facing flows.
- Instrument read routing in the data access layer, not in individual queries. A readable abstraction that identifies the appropriate connection (primary vs replica) based on request context is more maintainable than adding .useReplica() calls to every individual query. The routing logic belongs in one place.
Related reading
Frequently asked
The engineering bet behind Yashveer Labs
The bet I am running with Yashveer Labs is simple. Most software is built by people who treat it as a job. I treat it as a craft. Yashveer Singh, founder. Five production systems on the board so far. The arc points at machine learning, AI engineering, and cybersecurity. If your project is in any of those orbits, you are reading the right page.
Posts that line up with this one.
- SaaS Architecture and Scaling
Idempotency in API Design: Why It Matters More Than You Think
An idempotent API is one that handles repeated requests gracefully. Building it in from the start is far cheaper than retrofitting it after your first double-charge incident.
- SaaS Architecture and Scaling
Internal Admin Tools: Build vs Buy vs Retool
Every SaaS needs internal tools. The question is whether to build them, buy a platform like Retool, or use a lighter alternative. Here is the decision framework that saves engineering hours without creating tool debt.
- SaaS Architecture and Scaling
Job Failure Recovery: How Good SaaS Companies Sleep at Night
Every background job will fail eventually. The companies that sleep at night are the ones that built failure recovery into the system from day one, not as an afterthought when something broke in production.
- SaaS Architecture and Scaling
Monolith vs Microservices: Why Most Startups Get It Wrong
Microservices are the architecture that works at Netflix and fails at early-stage startups. Here is why the monolith is the right default, when microservices become rational, and how to make the transition without breaking everything.