WTF is a SuperColumn? An Intro to the Cassandra Data Model

http://arin.me/code/wtf-is-a-supercolumn-cassandra-data-model

by ropiku • 17 years ago

53 22 17 years ago

So it's not in first normal form. http://en.wikipedia.org/wiki/First_normal_form What's old is new again, and I wonder if they even know it.

What's not addressed is that this may not be an advance over Relational databases. In fact, when relational databases were first designed, databases like this were all-to-common, and RDBMS rejected this kind of data model explicitly, for good design reasons. For instance, how do you select from or join to parts of the complex values inside a column?

seiji• 17 years ago

I believe the answer is http://browsertoolkit.com/fault-tolerance.png

People are just trying to store data in the most efficient system possible. Using RDBMS software to store data only queried by primary key that you know will never need to be aggregated or dynamically queried can be overkill.

bhousel• 17 years ago

You have some catching up to do... :)

They are not at all concerned about normal form or joins - it's just not that kind of a database.

Here are some good blog posts that elaborate on the current non-RDBMS fad:

http://www.readwriteweb.com/enterprise/2009/02/is-the-relati...

http://www.metabrew.com/article/anti-rdbms-a-list-of-distrib...

http://randomfoo.net/2009/04/20/some-notes-on-distributed-ke...

http://bcbio.wordpress.com/2009/05/10/evaluating-key-value-a...

http://blog.boxedice.com/2009/07/25/choosing-a-non-relationa...

th0ma5• 17 years ago

The guy that invented the normal forms also went on to invent analytical forms, which are the exact opposite. For example, the Entity Attribute Value (EAV) model, used by many including the W3C Semantic Web.

blasdel• 17 years ago

It's too bad that RDF has been tainted by the W3C's pipe -dreams and been bludgeoned with a terrible XML serialization.

EAV is by far my favorite data model.

jbellis7• 17 years ago

> What's not addressed is that this may not be an advance over Relational databases

It's not. The relational model is much better, for many applications.

BUT.

The missing context is, the relational model cannot be sanely scaled out across multiple machines. Replication mostly works with some pain, but scaling writes is just a nightmare -- and what you have to do to partition your data across multiple rdbms nodes means giving up all that relational goodness. So what you end up with is neither of {easily scaled, relational}.

So if you can't have a scalable rdbms, the next best thing is a scalable key/columnfamily dbms. Something like Cassandra. Which, as a bonus, gives you significantly better per-node performance on modern hardware.

gruseom• 17 years ago

Which, as a bonus, gives you significantly better per-node performance on modern hardware

Could you expand on that last bit? i.e. what is it about modern hardware (and about Cassandra) that makes this true?

Maro• 17 years ago

Can you explain what the major advantage (and disadvantage) of the data model of Cassandra over a flat key-value store is?

harkham• 17 years ago

I'm curious: how easily does "easily scaled" mean? How much advance planning needs to be done about the eventual size of a cassandra cluster?

I think I can see how you can add machines when you need more capacity, but what about when you don't need all that capacity anymore? How do you go about removing machines from the cluster, and how does all of the nicely scaled out content get rebalanced when you do?

cosmohh• 17 years ago

looks pretty interesting. anyone already used it and maybe compared it with CouchDB, too ?

jbellis7• 17 years ago

Cassandra is a scalable key/columnfamily database designed for supporting low-latency applications with vast amounts of data partitioned across many machines. (Facebook has 40TB on a 150-machine Cassandra cluster.)

CouchDB is a document database that supports two-way replication so you can re-sync after taking part of the data offline, but it's still designed around the concept of a single master that holds all the data.

Completely different animals, in other words.

ddiljoy• 17 years ago

Does a Cassandra cluster stay write-available in the event of a network partition?

If so, how does it reconcile writes when the partition heals? Last I looked, Cassandra doesn't use vector/logical clocks - doesn't that potentially cause data loss when the partition heals if you're using a simple last-write-wins based on physical timestamps for a reconciliation policy? Does Cassandra use merkle trees for anti-entropy?

From what I can tell, although Cassandra claims to be write-fault-tolerant, the dependence on physical timestamps and the lack of the self-healing properties that merkle trees provide make me nervous about data loss and inconsistency when deploying it at scale.

omouse• 17 years ago

Facebook has 40TB on a 150-machine Cassandra cluster

So why did people stop using mainframes again?

damienkatz• 17 years ago

> CouchDB is a document database that supports two-way replication so you can re-sync after taking part of the data offline, but it's still designed around the concept of a single master that holds all the data

To clarify, CouchDB's replication features are peer-based and ad-hoc, there is no "master" replica. And you can take part or all of the data offline, or spread it geographically.

I think by single master, you mean CouchDB doesn't have support for partitioning a single logical database across machines, which is true (though there are projects for large scale partitioning with CouchDB, but it's not in the core yet).

WTF is a SuperColumn? An Intro to the Cassandra Data Model

Related Stories