We're Building Postgres in Rust. Using the LLVM of Databases

https://turso.tech/blog/a-new-modern-version-of-postgres-in-rust

by polyrand • 6 days ago

108 26 6 days ago

Author also discussing it on their Twitter [0]. I’m reserving judgement for now, but this has lots of potential usecases even apart from replacing postgres.

[0] https://x.com/glcst/status/2077759127682486561?s=46

cute_boi• 6 days ago

People said same thing about sqlite, but I love turso sqlite and haven't faced any issues.

egorfine3• 6 days ago

There is no turso sqlite

Omega359• 6 days ago

The llvm of databases is not turso, it's datafusion.

mhh__2• 6 days ago

Yes I assumed datafusion when I heard that phrase.

andriy_koval• 2 days ago

datafusion is llvmm for etl engines, they don't have indexes or transactions for example.

polyrand• 6 days ago

I'm already quite excited about Turso being SQLite-compatible, but adding many features on top.

And when a feature is not directly compatible with SQLite (ie: you can't directly read the file with `sqlite3`, it's straightforward to convert). This is great because you know you'll always be able to continue working with that database. Even if Turso stopped working, it's still a valid SQLite database.

A combination I would be excited about is:

- Full support for Postgres protocol/wire format (ie: Postgres, but in-process, backed by a single file). - Optional: Client/server architecture for further scaling and remote management using existing Postgres tooling - All backed by a SQLite-compatible file

They are already adding MVCC to SQLite anyway. So their effort seems doable, and I hope they succeed.

SOLAR_FIELDS3• 6 days ago

Supporting Postgres is a good goal but honestly the real challenge is extensions. Would supporting Postgres wire compatibility guarantee any Postgres extension would also work? This is one of the problems with Aurora - it’s all well and good until you need an extension that isn’t on the blessed list

lmz• 6 days ago

Would supporting Postgres wire compatibility guarantee any Postgres extension would also work?

It would not, for obvious reasons.

amluto• 6 days ago

There’s a section about extensions in the FAQ.

scotty79• 6 days ago

Will it have fast COUNT-s?

hoppp• 6 days ago

Sounds cool but also why? Is it so it's possible to have 1 database and then switch from sqlite to postgres without data migration?

khuey• 6 days ago

As someone who doesn't follow databases that closely, how is this different from Amazon Aurora (other than being open source)?

mhh__2• 6 days ago

Would be interested in strategic value add versus cedardb (although cedar is closed source in a way that at the moment I think may kill it)

andrea76• 6 days ago

Tried DOOM in that page, it's really slow...

mrdlm• 6 days ago

The ram is a blob on the db, ofc it's slow

dzonga• 5 days ago

If you have ever babysat a REFRESH MATERIALIZED VIEW cron job, or faked live views with a pile of triggers, read that sentence again: our views update themselves, live. This is the feature Postgres users have quietly wanted for twenty years.

Real-time materialized views - so far mostly Chinese tech shops have solved this - look at Apache Doris, StarRocks - but these are built on top of MySQL then maybe RisingWave which is Postgres wire compatible.

having real-time materialized views means you can take out 1 or 2 infra pieces from your stack e.g Flink & Kafka - for some simplified use cases. that means no more ETL jobs for some use cases.

We're Building Postgres in Rust. Using the LLVM of Databases

Related Stories