Druid – Distributed, real-time, analytical data store

by misframer • 12 years ago

88 9 12 years ago

gleenn• 12 years ago

Who "uses this in production" and how is it better than MySQL and Postgresql? They need a little more show instead of tell.

wesm• 12 years ago

It's used by Metamarkets in production (http://metamarkets.com/); I've seen talks about it at Strata and they're getting really impressive query times on terascale problems.

Shameless plug: if you're interested in fast in-memory analytics and more of the PyData bent, I'm building a system with some similarities to Druid at my company (http://www.datapad.io/) but more focused on optimizing single-node performance (i.e. minimizing EC2 expenses) on medium data (single columns of tables typically fit in memory) than high scalability / real time ingest.

stephengoodwin• 12 years ago

Netflix also uses Druid: http://www.slideshare.net/g9yuayon/netflix-druidstrata2013

markelliot• 12 years ago

It seems like it's part of Netflix's data pipeline, Suro[1].

[1] http://techblog.netflix.com/2013/12/announcing-suro-backbone...

fosk• 12 years ago

Take a look at this page: http://druid.io/druid.html#whois

calinet6• 12 years ago

To begin with, it's a column store, which has all kinds of cool implications for data that repeats itself a lot, such as analytics.

mratzloff• 12 years ago

Would it be fair to say that Druid is more suitable for known questions that are queried programmatically and mostly updated by time segment?

xvrl• 12 years ago

Druid was actually built to answer arbitrary questions that could not be answered with pre-aggregated data (because it can become computationally intractable), but still needed to be answered fast enough to power interactive dashboards. At Metamarkets, almost all queries are user driven through our interface. Currently Druid is indeed tailored for time-based event data, but it can shard on more than just time. Disclaimer: I am a Druid comitter and work at Metamarkets.

kposehn• 12 years ago

This is quite a great piece of kit. Definitely going to look into it for my own RTB uses.

One thing that strikes me as an immediate use is real-time impression bus aggregation for behavioral analysis. We ended up building our DSP primarily in Ruby+Redis/Postres and having this would have allowed some significant architectural changes.

Druid – Distributed, real-time, analytical data store

Related Stories