# ChatGPT Serves 800 Million Users From One Postgres Primary. Here's How.

800 million people use ChatGPT, and every write still lands on **one PostgreSQL primary**. No sharding.

When I read OpenAI's post *Scaling PostgreSQL to power 800 million ChatGPT users* by Bohan Zhang, I couldn't stop thinking about it. So I turned it into an interactive diagram: robots carry queries between the machines, and you can break things to see what happens.

👉 **Try it:** [mahmad.me/blocks/interactive/chatgpt-postgres](https://mahmad.me/blocks/interactive/chatgpt-postgres)

Here's the short version of how it works.

![](https://cdn.hashnode.com/uploads/covers/60a437cd59b7dc409c3d24e6/4b1337c6-495e-48ac-8dba-442f702d0fe0.png align="center")

## The problem

ChatGPT's database load grew **more than 10x in a year**. One primary handled every write and many reads, and incidents kept following the same pattern: something upstream spiked the load (a cache failure, a burst of expensive joins, a write storm). Queries slowed down, requests timed out, and the retries added even more load.

OpenAI fixed it in five moves.

## 1\. Move reads off the primary

![](https://cdn.hashnode.com/uploads/covers/60a437cd59b7dc409c3d24e6/8c1a665d-e97c-4f91-8d32-ebab08b2f905.png align="center")

Reads go to **nearly 50 read replicas** around the world, and the primary keeps the writes. Requests are also split into **high- and low-priority** replicas, so a heavy low-priority job can't slow down the important ones.

## 2\. Send write-heavy work somewhere else

Write-heavy work that can be sharded moved to **Azure Cosmos DB**. And new tables aren't allowed on Postgres anymore: new workloads start on the sharded systems.

## 3\. Pool the connections

Azure Postgres allows **5,000 connections**, and opening one was slow. **PgBouncer** keeps connections open and reuses them, so connecting dropped from **50 ms to 5 ms** on average.

## 4\. Stop cache stampedes

When a popular cache key expires, thousands of requests miss at once and all hit the database. With **cache locking**, one request refills the key and the others wait for it.

![](https://cdn.hashnode.com/uploads/covers/60a437cd59b7dc409c3d24e6/470bae51-e849-455a-aebb-eb96841f4f61.gif align="center")

## 5\. Protect the primary

- A **hot standby** is ready to take over if the primary fails, and reads keep working from the replicas meanwhile.
- **Rate limits** at several layers, all the way down to blocking a single expensive query pattern.
- **No giant joins.** One query joined **12 tables**, and its spikes caused high-severity incidents.
- **Only lightweight schema changes**, with a strict timeout.

![](https://cdn.hashnode.com/uploads/covers/60a437cd59b7dc409c3d24e6/6c85fe7f-bb23-478a-a0f1-8aadb9c8397b.gif align="center")

## The result

- **Millions of queries per second**
- **Low double-digit millisecond** p99 latency
- **Five nines** of availability
- **One SEV-0 in 12 months**, during the viral launch of ChatGPT image generation, when writes jumped more than 10x and over 100 million users signed up in a week

The lesson I took away: you don't always need to shard. A single primary goes very far if you keep everything you can off it.

## Play with it

In the diagram you can:

- switch between the **initial design** and **today's**
- trigger a **cache-miss spike** or take **the primary down**
- click any machine to **take it apart** and see what each part does

👉 [mahmad.me/blocks/interactive/chatgpt-postgres](https://mahmad.me/blocks/interactive/chatgpt-postgres)

All the facts come from OpenAI's post. The live numbers in the simulation are illustrative.