tiDB – Scalable RDBMS Inspired by Google F1 with Support for MySQL Protocol

by haisumbhatti • 11 years ago

88 32 11 years ago

onestone2• 11 years ago

How does it compare to CockroachDB?

tidb• 11 years ago

One of TiDB authors here. Our design is more similar to Google F1, currently we're focusing on building an independent SQL layer above any transactional KV storage.

ddorian436• 11 years ago

Does there exist any distributed transactional KV storage engine (open source) ?

onestone2• 11 years ago

So TiDB is similar to the (unfortunately) abandoned FoundationDB SQL Layer, but written in a proper language? Nice :)

samkone• 11 years ago

CockroachDB is closer to Spanner. The globally distributed primitives that power their SQL layer which is F1. I guess that TiDB is on layer above Cockroach.

BramG2• 11 years ago

Cockroach Dev here. We're actively working on a SQL layer actually. We really need to update our github readme to contain info about it. It's under active development and of course, it's inspired by F1.

BramG2• 11 years ago

Cockroach dev here. We're actually currently implementing a SQL layer within cockroach. So it wont' be layered, it's the main method of interaction with CockroachDB.

endymi0n• 11 years ago

Seems like AWS Aurora just got an open source competitor.

I'm just a little sad that it's all MySQL again, now that we're betting on PostgreSQL and don't really look back.

On the performance side I'm just a litte worried however. While Go is an excellent choice for performant and concurrent app-level code, I'm not too bullish on the language on the database level. InfluxDB sucked hard when we put it under some more load, but let's see what comes out there.

ngaut3• 11 years ago

PostgreSQL support is on our roadmap.https://github.com/pingcap/tidb/blob/master/ROADMAP.md

c4pt0r2• 11 years ago

We choose MySQL protocol first because it's widely used and we are more similar with the tool chain and protocol in comparison with PostgresSQL. About the performance, Golang is design for building distributed system in Google, and the development productivity is perfect, InfluxDB's performance problem is partly caused by other reason.

creshal• 11 years ago

I'm just a little sad that it's all MySQL again, now that we're betting on PostgreSQL and don't really look back.

Doesn't Postgres have its own multi-master solution in the works with BDR?

ddorian436• 11 years ago

bdr is async replication and doesn't support sharding

sagichmal• 11 years ago

InfluxDB's performance problems have very little to do with the implementation language, quite a lot to do with fundamental architectural and team constraints.

ericb• 11 years ago

Can you detail, or point me to a blog post which details this stuff? As someone who is about to invest a lot of time in researching use of influxdb at scale, I'm interested in doc of the performance problems, and even moreso if the performance is troubling in the long term due to the architecture or team. Anything you can point me to will save me a lot of time.

ddorian436• 11 years ago

aws-aurora doesn't support sharding/scaling

acdha3• 11 years ago

Can you expand on this a bit with some actual technical details about how you think it doesn't scale enough?

danbruc• 11 years ago

Could one of the authors provide some more information about the goals and architecture? Do I understand it correctly that the goal is to implement a relational database on top of one of a couple of different key value stores and aiming, among other things, for drop in compatibility with MySQL? What are expected benefits?

EDIT: The architecture diagram wasn't visible to me before, now that it suddenly appeared things are way more clear.

ngaut3• 11 years ago

The goal of TiDB is to create fault-tolerant distributed RDBMS. Especially for distributed transaction support. Let developers get benefits without changing any of their existing code.

brandtg• 11 years ago

I'm curious to know the motivation behind publicizing the project at this stage in development, as it seems like the key feature (distributed transactional storage engine) is quite far away on the road map.

Are there any design documents detailing its implementation? I checked the wiki but it didn't look like there was anything there. What alternatives were considered, and why were they abandoned?

Also, is there a concrete use case for which this system is being built? If so, what are some (publicly releasable) details about the use case, e.g. access patterns, data volume, etc.?

coocood• 11 years ago

We open-sourced on early stage because we want to get feeds back from the community and build the project with community involved.

We will consistently deliver new features like HBASE support, and the documentation will be improved as well.

manigandham• 11 years ago

So a SQL layer on top of a KV base? Isn't this very similar to what FoundationDB did (and had a lot of trouble with)?

if so, good post by John Hugg of voltdb about it: http://voltdb.com/blog/foundationdbs-lesson-fast-key-value-s...

How does tiDB compare to this?

nickpsecurity• 11 years ago

Darnit, acquired by Apple and shutdown!? I didn't even know! Thanks for the link. His critique is actually the first I've seen on their tech. He speculates quite a bit despite them giving plenty of detail and claiming to use many of same choices as Google's F1. If they failed, I'd love to know in detail how to improve on their successes. Some of his gripes have obvious work-arounds with others probably having work-arounds.

That said, I think props still go to Google's teams on GFS, Spanner, and F1 as the best techs out there. A F1 variant, which two teams are trying, is best approach given it's proven and with much published detail.

no1youknowz• 11 years ago

Looks good, especially the PostgreSQL support coming up.

What about MPP (Massively Parallel Processing) support? This is one of the things that make CitusDB so powerful.

skyde• 11 years ago

Please correct me if I am wrong, but it look like it is not distributed yet. From the roadmap : https://github.com/pingcap/tidb/blob/master/ROADMAP.md Distributed KV and distributed Transactions have no checkmark! So in it's current state, how does that differ from using sqlite?

tiDB – Scalable RDBMS Inspired by Google F1 with Support for MySQL Protocol

Related Stories