Postgres locks do not scale

https://www.recall.ai/blog/postgres-locks-do-not-scale

by timetoogo • 13 days ago

25 8 13 days ago

d4v3• 12 days ago

Separate high-frequency operational state from large, flexible, searchable metadata. It is bad data models that don't scale

hylaride• 12 days ago

with an unusually large load we surpassed our provisioned IO on the underlying volume

Sounds like they could have benefitted from Aurora's storage model (though there are reasons not to use it, too).

sgarland• 12 days ago

How people are running any RDBMS in prod — but especially Postgres - and not using a connection pooler with the capability to pause incoming connections is beyond me.

The entire “how do we drain connections” issue described in TFA becomes trivial, as does gradually raising the amount of incoming connections, let alone the load advantages from multiplexing the client connections.

CubsFan1060• 12 days ago

15,000 database connections seems like a lot. Is that a scale Postgres supports very well?

jbonatakis• 12 days ago

That’s well beyond what a stock Postgres server would reasonably handle

tony_cannistra• 12 days ago

Yeah. This is why we have connection pooling.

nosefrog• 12 days ago

Don't reboot the db during your next outage.

nasretdinov• 12 days ago

As others have already pointed out, the issue is too high concurrency — IMO that should be the main outcome of the analysis. E.g. InnoDB has innodb_thread_concurrency setting that applies for internal operations to prevent exactly kinds of issues like these. Yet MySQL isn't perfect in any regard either, it's also very much possible to hit lock (or latch as they're called in DB world) contention if you hit the same rows in a table too hard from too many different threads. It's a fundamental limitation of the CPUs that they can't do too much of parallel read-modify-write, and it starts to break down very quickly with the number of cores

Postgres locks do not scale

Related Stories