HN user

std_reply

33 karma
Posts0
Comments13
View on HN
No posts found.

Go is used for the SQL layer, it’s a modern optimizer that can do distributed joins etc. ie. Issue parallel reads to the storage nodes.

Additionally, it can push down the DAG to the TiKV storage nodes, written in Rust, to reduce movement of data and work closer to the physical data.

I would argue the opposite, distributed databases are much easier to operate at large scale. Truly online distributed DDL, at least in TIDB, strong consistency etc.

People who bang on about Postgres replication have rarely setup replication in Postgres themselves and that too in the 100a of Pb scale.

MySQL replication works well and can be scaled more easily (relative to Postgres) but has its own problems. eg., DDL is still a nightmare, lag is a real problem, usually masked by async replication. But then eventual consistency makes the application developers life more complicated.

TiDB is running core banking services too.

I think people’s idea of scale and operating at scale is limited to their experience.

You can get MySQL to run at any scale, look at Meta and Shopify. Operational complexity at that scale is a different story.

Distributed databases reduce a lot of the operational complexity. To take one example:

Try a DDL on a 5 TB table in any replicated MySQL topology of choice and compare it with TiDB’s Distributed execution framework.

TiDB has four main components:

1. SQL front end nodes 2. Distributed shared nothing storage (TiKV) 3. Meta data server (PS) 4. TiFlash column store

1 and 3 are written in Go 2 is written in Rust and uses RocksDB 4 is written in C++

2 & 3 are graduated CNCF projects maintained by PingCAP.

Disclaimer: I work for PingCAP

AirBnB, Databricks, Flipkart 3 of the largest banks in the world, some of the largest logistics companies in the world, at least 2K seriously large installations.