Ex-Googlers CockroachDB: A Scalable, Geo-Replicated, Transactional Datastore

https://github.com/cockroachdb/cockroach/blob/master/README.md

by ethanjones • 12 years ago

98 66 12 years ago

rabino• 12 years ago

Why the 'Ex-Googlers' in the title. Is it like a seal of approval or something?

Yeah, I feel a bit uncomfortable about using the brand of a previous employer so prominently to promote a different project

quotemstr• 12 years ago

Haven't you noticed the pedigree obsession of some in our field?

wslh2• 12 years ago

In this context it adds more signal than noise. There are zillions of open source projects but when you need to use them in production a very small subset of this universe is ready.

angrymouse• 12 years ago

I think that refers to the Wired article http://www.wired.com/2014/07/cockroachdb/

With the tag being sign of pedigree. "A scalable DB from people who worked at a place with huge scale DBs".

Whether it is very convincing or not is another matter, but I am sure it gets more clicks/attention

eng_monkey• 12 years ago

It is a pretty strange title. Some might interpret it as they started this project after being fired from Google.

bojo2• 12 years ago

That's kind of how I read it.

melling2• 12 years ago

Yes, it's called social proof. If you're observant you'll notice that people use it everywhere. Of course, social proof doesn't mean guarantees.

peterwwillis2• 12 years ago

Nah, it's honor-by-association fallacy. Social proof is a behavioral thing, not false reasoning.

datashovel4• 12 years ago

I don't think it matters how you try to promote an open source project. The code will ultimately be the determining factor whether the project can support a following. And everyone has access to the code to make that determination independently.

acjohnson55• 12 years ago

That seems rather naive to me. No one is scouring GitHub looking for quality code and bringing unrecognized projects to light.

acveilleux• 12 years ago

A requirement of a successful project/product is visibility. Ex-Googlers does not imply much more then "used to work there" but if it drives more attention to them, it's good MarCom.

kfcm• 12 years ago

Name-dropping has been common since the first salesman.

Blackthorn• 12 years ago

This was based on an internal Google technology, so the "Ex-Googlers" label is actually quite relevant.

ryandrake• 12 years ago

I was kind of put off by this as well. Does where they worked in the past have anything to do with how useful the project is now or how talented the people are? What if they're doing it on their own because Google thought the project was useless? What if they were fired from Google, or only worked at Google for a month? How does the founders' past involvement with Google mark this project as any more interesting than any other random "Show HN" project?

ultimape• 12 years ago

I hope Ceph can takes cues from this and be able to do geo-replication at scale.

srcmap• 12 years ago

Comparing to Facebook ' s TAO approach from yesterday post, I like the FB small set of api approach better. But that mainly focus on handling social graph with objects and associations.

On the other hand, almost all my backend related features can be easily abstracted to those APIs.

Shish2k• 12 years ago

Comparing to Facebook ' s TAO approach from yesterday post

Link? I see no relevant-looking "Facebook" or "TAO" in my recent RSS entries :(

anonetal• 12 years ago

I think he is talking about this post: https://news.ycombinator.com/item?id=8557408

IgorPartola2• 12 years ago

So I cannot tell if this is aiming to be CA or AP. Having beaten my head against the CAP wall for a while, how does it deal with partitions?

pjc502• 12 years ago

The design doc https://docs.google.com/document/d/11k2EmhLGSbViBvi6_zFEiKzu... says, "TBD: how to avoid partitions? Need to work out a simulation of the protocol to tune the behavior and see empirically how well it works"

.. so they've gone for CA and forgotten about P.

e12e• 12 years ago

Georeplication that can't handle partitions? Sign me up! I'll have two (or more!)!

rdtsc• 12 years ago

Usually when building a distributed database, "TBD: how to avoid partitions?" is not what you want to see except on the initial whiteboard discussion before a single line of code is written. But what do I know, I am not an Ex-Googler.

teraflop3• 12 years ago

You're taking a sentence out of context and giving it the most uncharitable possible interpretation. They certainly haven't "forgotten" about network partitions because if you actually read the design document, instead of ctrl-F'ing for the word "partition", they talk about the mechanisms they use to ensure sequential consistency. The software is not yet at the point of being testable AFAIK, but clearly the intent is to build a CP system.

The section you're quoting is discussing a separate gossip protocol that is used to lazily propagate node state information. It does not affect the consistency of actual data replicas.

grogers• 12 years ago

First off, you can't have CA in a distributed system, but I'm guessing that was a typo and you meant CP or AP. Given that it is based on spanner, and claims to have something close to linearizability, it is CP.

curiousDog• 12 years ago

From what I understand, in real world distributed systems, you will always have P (despite what the name suggests, it's pretty much failures not just network partitions). The question is whether they'll choose A over C when a partial P happens (say you have 3 nodes and 2 go down, you have quorum loss, will you still keep the replica available and replicate lazily or will you make it unavailable until a 2nd replica comes up?). I'm guessing it'll be tunable/configurable in this case.

datashovel4• 12 years ago

Happy to see it's written in Golang.

bojo2• 12 years ago

As a Go developer I was happy as well.

On the other hand this question immediately popped into my mind: What kind of overhead does the GC incur, and how does it affect processes like a database where low latency is desired?

datashovel4• 12 years ago

If I had to guess, I would say network and disk I/O will be the bottlenecks. Disk I/O less so because distributed system, and SSD. I imagine if anyone can solve GC issues, though, I would bet on Google :)

ansible• 12 years ago

What kind of overhead does the GC incur, and how does it affect processes like a database where low latency is desired?

The eventual goal (golang v1.5 IIRC) is to have 40ms out of every 50ms available for actual processing. This is the kind of 'soft real time' that should provide good responsiveness for most clients most of the time.

tedchs• 12 years ago

Here in the Southeast, we call them "Palmetto Bugs". Maybe a rename to PalmettoDB? :)

viiralvx• 12 years ago

South Carolina? :P

wuliwong• 12 years ago

Palmetto bugs are generally the larger, flying "American Cockroach." "German Cockroaches" are smaller, don't fly but are generally the ones that cause infestations. I grew up in the Northeast and German Cockroaches are all I ever saw up there. I've been down in Atlanta for years now and I only see Palmetto bugs here.

Unless this database can fly, I'd say they've chosen the right name. ;)

JackC2• 12 years ago

Are there distributed data stores like this that are also resilient to intentional sabotage?

I've been looking recently at long-term digital preservation systems -- tools designed to archive large amounts of data for decades. This is the Library of Alexandria problem -- how do we preserve all this data we're generating against once-in-a-century disasters?

So this 2005 paper lists thirteen different threats to long-term archives: Media Failure; Hardware Failure; Software Failure; Communication Errors; Failure of Network Services; Media & Hardware Obsolescence; Software Obsolescence; Operator Error; Natural Disaster; External Attack; Internal Attack; Economic Failure; Organizational Failure.[1]

Fault-tolerant distributed data stores are exciting, because they solve a bunch of those problems off the bat -- media failure, hardware failure, communication errors, failure of network services, hardware obsolescence, and natural disaster.

They also help to address software failure, software obsolescence, and economic failure, because archival projects are always strapped for resources and it's great to rely on tools that exist for totally distinct, commercially-valuable reasons.

But that still leaves operator error, external attack, and internal attack -- burning down the Library.

Hence my original question: are there distributed data stores that can be configured to resist intentional destruction of data?

[1] http://www.dlib.org/dlib/november05/rosenthal/11rosenthal.ht...

otoburb• 12 years ago

Journaling or storing incremental backups (perhaps offline?) of validated/verified checkpoints may address this, although it sounds like something you wouldn't be happy with since it's not a 'built-in' feature but an additional backup & maintenance process that a system administrator would need to implement.

I guess you're asking whether there exists a distributed fault-tolerant with a form of version control (similar to git/cvs/perforce) as part of the native feature set.

imaginenore• 12 years ago

are there distributed data stores that can be configured to resist intentional destruction of data?

Well, Git has checksums on everything.

jchrisa• 12 years ago

Maybe Camlistore? https://github.com/bradfitz/camlistore

tomp• 12 years ago

The problem with internal/external attacks is that we (the society) don't really want to prevent it. The reason is simple: child porn. To date, Bitcoin block chain (and related ideas) is the only data-storage that is 100% resistant to attacks (i.e. changing history), but luckily it cannot handle amounts of data large enough to be viable for child porn (or most other forms of media). Tor, on the other hand, gets a bad rep precisely because it doesn't prevent it (despite its numerous other, beneficial, uses).

The core of the issue is that humans view different information differently (child porn vs. Mona Lisa), whereas for computers, bits are bits and numbers are numbers. As long as child porn remains illegal and socially unacceptable, we'll want to enable attacks on data, i.e. for someone (usually internal operators) to be able to delete some kind of information, corrupt it or at least track it. Of course, this necessarily means that all information stored in the same data-store will be vulnerable.

mhb• 12 years ago

You're conflating the archival properties of the medium with the decision about what to save. Oil paint on canvas is durable. It doesn't mean that a museum needs to retain every piece of crap that anyone paints.

sciurus• 12 years ago

Any idea how well LOCKSS would handle this?

http://www.lockss.org

http://blog.dshr.org/2014/07/trac-certification-of-clockss-a...

JackC2• 12 years ago

LOCKSS is definitely a giant in this field, and David Rosenthal (who wrote the paper I linked as well) is great.

But LOCKSS occupies a small niche. My hope is really that at some point a commercially-focused project with a ton of engineering effort and battle testing behind it will displace a lot of what LOCKSS has had to do manually. Seems like that might happen as web services get more and more distributed and fault-tolerant.

kul_• 12 years ago

awful name!

eksith2• 12 years ago

I think the sentiment was "hard to kill" and/or become extinct due to its resiliency. But I agree, they should have picked something else.

vram22• 12 years ago

Good point about the likely reason.

Hydra could have been another good choice :)

http://en.wikipedia.org/wiki/Hydra

See first entry at above Wikipedia page, about the many-header serpent.

disputin• 12 years ago

I think the name's great. A refreshing change from all the synthetic cutesy crap. Bubblegumlydb?

Thaxll• 12 years ago

Many things that I don't agree with: https://github.com/cockroachdb/cockroach/

MySQL: Weak consistency

Cassandra: No availability or weak consistency with datacenter failure

teraflop3• 12 years ago

MySQL provides strong consistency when run on a single machine, but that breaks down when you handle failover using asynchronous replication between DCs.

eksith2• 12 years ago

To add to this "Postgres: Limited scalability". I wish someone told us before we ticked over to 31TB... 5 Months ago.

JSno• 12 years ago

the name feels sick

vishly2• 12 years ago

Awful name!!! Why Cockroach..??

vishly2• 12 years ago

awful awful name... folks didnt get anything than cockroach..

peterwwillis2• 12 years ago

It's really strange to see networked databases go through this iterative design fad. It was file transfers back in the 90s/00s; everyone had their own distributed decentralized file transfer solution. Little known fact, Gentoo's Portage almost became an internet-wide distributed decentralized public file system. Thank god they abandoned that idea. Can you imagine trying to debug a file transfer error just to get an mp3 player to install on your machine? I can't wait until databases go back to being mainframes.

anonbanker• 12 years ago

Little known fact, Gentoo's Portage almost became an internet-wide distributed decentralized public file system.

Am I the only person really sad that this didn't happen? after using apt over Tahoe-LAFS (over I2P - KillYourTV's PPA is on clearnet and I2P), I wanted this to be the default behavior for apt.

xedarius• 12 years ago

Are there any details on how the distributed joins are achieved? (Sorry if that detail is in the design doc, my access to google drive is blocked from work).

Ex-Googlers CockroachDB: A Scalable, Geo-Replicated, Transactional Datastore

Related Stories