Our take on RethinkDB vs. MongoDB

http://www.rethinkdb.com/blog/mongodb-biased-comparison/

by coffeemug • 13 years ago

116 83 13 years ago

lucian19003• 13 years ago

It does indeed look very much like MongoDB, but made by people that actually know what they're doing. It's refreshing to see good database design for a change.

dwnoble• 13 years ago

Can you expand a little bit on this? What design decisions in MongoDB vs RethinkDB are you referring to?

lucian19003• 13 years ago

Rethink has durability, MVCC, joins, logical sharding, excellent admin tools, etc. All things that serious databases tend to have and Mongo doesn't.

wiremine2• 13 years ago

Most of the posts I've seen about RethinkDB focus on "hey, we're a better NoSQL solution than MongoDB." That could be true, but so far I see it mostly coming from RethinkDB themselves, or people who like the design in theory.

However, does anyone have any practical real-world experience using it? It's not production ready (from what I gather), but has anybody actually used it for real world stuff?

For my own part, I tried it out, and got stuck trying to implement a many-to-many style join. I did some searching, and it looks like that is not really possible at this point. Not a bit deal, but it might be handy to have some example SQL-to-RethinkDB queries, just to help us newbies figure out the ropes.

coffeemug17• 13 years ago

Hi, slava @ rethink here. People have been using Rethink for lots of projects, but there are still some showstoppers we have to work out (better docs notwithstanding) -- remember, the product has been in the wild for less than 90 days.

What did you try to do with a many-to-many join? We could help you with writing the query, and could add syntax sugar to the language to make it easier if it makes sense.

wiremine2• 13 years ago

> "remember, the product has been in the wild for less than 90 days."

I hear you, and, FWIW, I'm excited about Rethink. To rephrase my question/observation: your article clearly lays out why you think it is better than MongoDB, using some quotes from people who agree with you. However, without some real-world data, it is still an argument rooted in theory. I like theory, but I also like to take real-world data to my bosses. Do you have any stats/examples that actual compare and demonstrate the performance? (I understand that wasn't the purpose of your article, just asking as a follow up).

Regarding the many-to-many joins: I was just playing around with a contrived example: "a blog post has and belongs to many categories." Mainly I was just curious how to do it, I didn't _need_ it for anything. But, I couldn't figure out how to write it with the query language. I was using Python DSL, FWIW.

jdoliner• 13 years ago

A many to many join should be possible in RethinkDB using `innerJoin`. What did your data schema look like?

alexpopescu2• 13 years ago

> However, does anyone have any practical real-world experience using it? It's not production ready (from what I gather), but has anybody actually used it for real world stuff?

There are people that told us they are experimenting with it for real apps, they've sent extensive feedback (most of these can be seen on GitHub), and some have started to build libraries for RethinkDB. As with anything young and open source, it's difficult though to tell with certainty how many projects are using a tool and what stage are they.

> it might be handy to have some example SQL-to-RethinkDB queries, just to help us newbies figure out the ropes.

Working on it already.

alex @ rethinkdb

islon2• 13 years ago

Every NoSQL database is perfect and better than all the other options until you start using it in the real world. I'm not saying rethink DB is not a good solution, the point is, nosql dbs are about compromise and specific problems.

coffeemug17• 13 years ago

Hi, Rethink is my baby. It is beautiful, but it is not perfect :)

TY2• 13 years ago

Off topic: Slava, I wish you could resume writing for your blog - you've always had interesting things to say.

islon2• 13 years ago

I wasn't criticizing rethinkDB, only the "marketing speak" or excessive excitement that normally comes with every new nosql solution. Anyway, I wish I had the skills to create such a complex piece of software, congrats =)

film422• 13 years ago

I agree. I think NoSQL began exploding when people started doing more custom work and pushing further away from big orm based frameworks because of NoSQL's relaxed schemas (or lack thereof) making it easier on developers. I still really like NoSQL for those reasons, but like everyone knows, in the real world there are a lot of gotchas, which seem to come from the designers trying to solve one purpose (many specific DBs) which makes sense considering the yet uncharted territory of NoSQL (proof of concepts). I'm excited to see how RethinkDB fits into the real world though. It looks as though someone went back on years of research across multiple experiments and took what was right and built it. But that said, just because one shirt is really nice and a pair of pants fits amazing doesn't mean they match, and that's why I agree with you.

desireco423• 13 years ago

I started using RethinkDB in one of my projects and am looking for excuses to use it in more of them. So far things have been great and honestly my impression is that RethinkDB doesn't get nearly the hype it deserves.

I used Mongo before and it is fine db and I don't think I would be sad to use it, however rethink really does so many things better.

Again I just started using it and things are really good, I didn't ran into any obvious limitations and annoyances.

There are several features that I really like, for example: web admin is really well done, it is easy and obvious how you create cluster, there are a lot of small things that made me jumpstart my development faster, as I can run queries in admin to try them out and I also get data back to see how things will look like.

The only thing I am somewhat missing is 'brew install rethinkdb'

film422• 13 years ago

I know it's not formal for a comment like this, but: +1 for 'brew install rethinkdb'

coffeemug17• 13 years ago

Actually, 'brew install rethinkdb' is now available. I'll update the install page with it in a moment.

EDIT: https://github.com/rethinkdb/rethinkdb/issues/269

desireco423• 13 years ago

Thank you :), again it is not really serious gripe as regular installer does the job just fine, but I tend to install all apps like that and this helps.

+1 for sure for this one

btw, when I was installing it, I tried that even before checking on homepage :)

desireco423• 13 years ago

I would like to add (not to edit original post one more time), that in one of those things I like is query language. I remember how it felt weird initially doing simple things in Mongo, language is very simple and queries make sense and are easy to read.

primitur2• 13 years ago

Why brew instead of port? I've always wondered why the pendulum swings between ports and homebrew so often on HN ..

zwily• 13 years ago

I don't think that pendulum has swung to the MacPorts side for many years.

bfirsh• 13 years ago

This just reads like marketing speak. What are the disadvantages of RethinkDB?

coffeemug17• 13 years ago

http://www.rethinkdb.com/docs/faq/#when-is-rethinkdb-not-a-g...

http://www.rethinkdb.com/docs/advanced-faq/#is-rethinkdb-imm...

bcbrown• 13 years ago

You mention comparing against Hadoop for computationally-intensive data analysis. Would Rethink be suitable for a several-terabyte dataset with non-computationally-intensive analytics?

Currently we're using Hive and Python over streaming Hadoop. There's no significant ongoing data accumulation; we're just analyzing the data we have.

alexpopescu2• 13 years ago

personal != marketing

P(biased|personal) -> 1

optimiz34• 13 years ago

My company uses MongoDB. Our biggest pain points are:

1. MongoDB has massive storage overhead per field due to the BSON format. Even if you use single character field names, you're still looking at space wasted on null terminators. 32bit fixed length int32s also bloat your storage use. We solve this by serializing our objects as binary blobs into the DB, and only using extra fields when we need an index.

2. In Mongo, the entire DB eventually gets paged into memory and relies on the OS paging system which murders performance. For a humongous DB, not so much.

3. #1 and #2 force #3, which is sharding. MongoDB requires deploying a "config cluster" - 3 additional instances to manage sharding (annoying that the nodes themselves cannot manage this, and expensive from an ops/cost standpoint).

What I would like to know is:

1. What is the storage overhead per field of a document in RethinkDB? If it's greater than 1 byte, I'm wary.

2. Where is the .Net driver?

coffeemug17• 13 years ago

1. In the coming release we'll be storing documents on disk via protocol buffers, which, unlike BSON has an extremely low overhead on fields. A few releases after that we'll be able to do much better via compression of attribute name information (though this feature isn't specced yet).

2. No ETA yet, but we're about to publish an updated, better document, better architected client-driver to server API spec, so we'll be seeing many more drivers soon.

optimiz34• 13 years ago

If you use proto-bufs, it means you already have a system for internal auto-schematization. Why not pack all the fields together and use a bit-vector header to signify which fields are present and which fields have default values? I'd LOVE to see a document DB with ~1 bit overhead per field.

ukd15• 13 years ago

#1 - you might find it's not just the per field overhead, but the per document one. Check out the powerOf2Sizes settings;

http://docs.mongodb.org/manual/reference/command/collMod/#us...

10gen have been thinking about compression but nothing specific has happened yet (https://jira.mongodb.org/browse/SERVER-164). ZFS + compression is interesting, but not 'production' quality if you're using linux, and last time I tried to get MongoDB running on Solaris I gave up...

optimiz34• 13 years ago

What really bugs me about Mongo:

https://jira.mongodb.org/browse/SERVER-863

The issue has been open for over two and a half years, is one of the most highly voted issues, and has yet to even have reached active engineering status.

Agree with you that compression is just a workaround for the awful BSON format.

DonnyV2• 13 years ago

.Net driver is right here. https://github.com/mongodb/mongo-csharp-driver/downloads

optimiz34• 13 years ago

I meant for RethinkDB (BTW - I've been a contributor for the MongoDB driver).

rubyrescue3• 13 years ago

Riak is NOT operations-oriented. It's nearly impossible to manage operationally without dedicated staff at scale and the tools to introspect and analyze and deal with failures aren't robust enough yet.

I know they're just trying to contrast Riak and Cassandra with Couch and Mongo, and that Riak is designed to shard easily without the developer having to think about it.

That philosophy actually is "developer-oriented" in that it SEEMS like an operational savings because it was designed by developers.

coffeemug17• 13 years ago

That's an interesting take -- I think the Basho team positions Riak as an operations-oriented system (at least that's how I always thought about it, though I haven't used Riak in production).

rbranson• 13 years ago

Can you provide an example of where Riak was "nearly impossible to manage operationally without dedicated staff at scale and the tools to introspect and analyze and deal with failures aren't robust enough yet"?

rubyrescue3• 13 years ago

Just watch the talk by Voxer on Riak (on Basho's site). It's basically an hour long explanation of why the things that come out of the box with Riak don't work for them. We ran into those problems as well.

Also, I'm giving a talk on it in 9 days at ErlangDC...Whisper is now top 10 social networking apps and we had a number of critical Riak failures. i'll be elaborating on them, though the focus of the talk is not to bash Riak, no pun intended... just to provide our experience and how we worked around it.

argvzero2• 13 years ago

Chief Architect at Basho here:

Saying Riak is categorically non-operations-oriented is a bit hyperbolic, but I will be the first to acknowledge that we need even more visibility into failure-recovery / degraded mode situations. I've spoken to a few customers who have "cheat sheets" of Erlang console commands they use to debug things like handoff slowness or poor performance in general. This alone means we need to do better,

On the other hand, Riak continues to function in scenarios where other databases would be completely unavailable. I'll take immature visibility during those situations over complete unavailability any day,

I appreciate your feedback - I can assure you that this is something we're constantly working on and you'll see improvements with each release.

Finally, if you've been bitten by anything specific you'd like to see fixed, we do all our development in the open at http://github.com/basho, so github issues, pull requests, etc go right into our internal tools and workflows.

Cheers,

Andy Gross

rubyrescue3• 13 years ago

Would love to give some feedback. I like Riak and I believe in it - over the long haul I'm 100% sure it will be an amazing product.

Let's grab beers at Erlang Factory!

Chad

munimkazia• 13 years ago

We have been looking at NewSQL(or even NoSQL) platforms for our databases at my place of work, and we also stumbled upon RethinkDB. While everything these guys say does sound amazing, we were looking for someone who has implemented it, or any third party case study about it. Since we couldn't find any, we decided not to go with RethinkDB for now.

Does anyone here know any big website/service which uses RethinkDB?

apendleton• 13 years ago

It's brand-new, and not recommended for production use yet, so I highly doubt that anything like that exists.

mhd• 13 years ago

How's the general performance and memory consumption on smaller machines, e.g. entry-level VPS's or the lower spectrum of AWS VMs? Don't have any big projects in the pipeline that immediately required sharding etc, but would like to play with it on a few weekend-scale items.

coffeemug17• 13 years ago

We did some testing and it should be great, especially once https://github.com/rethinkdb/rethinkdb/issues/97 makes it in.

ukd15• 13 years ago

When I last tested I maxed out at about 700 inserts / sec with nothing else happening (MBP Retina, SSD, etc) - it's not bad, but not as fast as Mongo.

I'm going to benchmark it when I get some time!

mintplant• 13 years ago

I'm eager to take RethinkDB for a spin, as soon as secondary and compound indexes are fully implemented.

[1] https://github.com/rethinkdb/rethinkdb/issues/88

[2] https://github.com/rethinkdb/rethinkdb/tree/jd_secondary_ind...

jcdavis• 13 years ago

Not mentioned: RethinkDB doesn't yet support secondary and compound indexes, which is a dealbreaker for a lot of setups

Definitely looks interesting though, and I look forward to playing around with it at some point.

andrewljohnson• 13 years ago

Read the article:

"Some key features like secondary indexes and live backup are still in development"

coffeemug17• 13 years ago

Sorry, we added limitations a few hours after posting. I should have been more clear about the fact that they were edited in. I'll try to get better at this live blogging thing :)

dennis82• 13 years ago

this is marketing cloaked in a developer portal. I think it's great that rethinkdb is trying to distinguish themselves from Mongo, but what's the real marginal utility of a rethinkdb over Mongo?

Mongo has been around for years, and it still has problems.

Rethinkdb is just launching a new product that essentially does the same thing as Mongo, but is maybe just a little easier to use.

I think the Yet Another Database (YAD) question still hasn't been answered by this post.

etanol• 13 years ago

    «An asynchronous, event-driven architecture based on 
     highly optimized coroutine code scales across multiple
     cores and processors, network cards, and storage systems.»

It may be a dumb question, but isn't this statement a bit contradictory? As far as I understand, event-driven design and coroutines (i.e. cooperative multitasking, lighweight threads, etc.) are the techniques usually chosen to AVOID concurrency.

How does such a design imply multicore scalability? Obviously, coroutines and event loops don't prevent you from running in multiple cores. I just fail to see the correlation.

coffeemug17• 13 years ago

This is a great question. We start a thread per core, and multiplex thousands of coroutines/events on each thread. When coroutines on different threads need to communicate, we send a message via a highly optimized message bus, so cross-thread communication code is localized. This means each thread is lock-free (i.e. when a coroutine needs to communicate with another coroutine, it sends a message and yields, so the CPU core can process other pending tasks). The code isn't wait-free -- a coroutine might have to wait, but it never ever locks the CPU core itself. So, as long as there is more work to do, the CPU will always be able to do it.

If instead we used threads + locking like traditional systems, we'd have to deal with "hot locks" that block out entire cores. Effectively we solved this problem once and for all, while systems that use threads + locks (like the linux kernel) have to continuously solve it by making sure locks are extremely granular.

JulianMorrison• 13 years ago

Sounds very Erlang-ish. Did you copy that deliberately?

shrughes• 13 years ago

You get less overhead by using less often the low-level concurrency primitives that involve cross-core synchronization. Cross-core synchronization happens in rethinkdb mainly when you see an on_thread_t object constructed or destroyed (and also in a few other places) and those get batched when you have more than one per event loop (which is not necessarily good, inflated queue sizes is also something to be wary of). So if you want to attach a bunch of network cards and high-speed storage devices on opposite ends of a handful of CPUs, your throughput won't be hindered by the fact that millions of threads are trying to talk to one another.

bunkat2• 13 years ago

Looks very interesting, but this statement in their FAQ is a red flag for me:

How can I understand the performance of slow queries? Understanding query performance currently requires a pretty deep understanding of the system. For the moment, the easiest way to get an idea of why your query isn't performing well is to ask us.

Wish RethinkDB was a little further along because it seems like it might be a good fit for a new service I'm building.

neumino2• 13 years ago

Michel @ RethinkDB

We are building a tool to explain in a nice way how the query is executed, what are the bottlenecks etc. It should make it for 1.5.

You can track progress here https://github.com/rethinkdb/rethinkdb/issues/175 (it's kind of empty for now)

bunkat2• 13 years ago

Interesting, I'll keep an eye on that. When do you think RethinkDB will be ready for production use?

raxitsheth• 13 years ago

geo support? i think No!

arunoda• 13 years ago

This comparison speaks better than this - http://www.rethinkdb.com/docs/comparisons/mongodb/

smagch• 13 years ago

Other than RethinkDB, BigCouch looks both Developer/Operation oriented database since it is a Dynamo-like CouchDB. Does anyone have a BigCouch experience?

primitur2• 13 years ago

Anyone know if there is a Lua driver for RethinkDB in the works? I suppose I could use the C client and think about generating one for Lua, but maybe someone has already done that?

coffeemug17• 13 years ago

There isn't one yet. We'll publishing the new, much simpler spec for client driver writers. It'd be pretty easy to do a native Lua driver (based on protocol buffers).

DonnyV2• 13 years ago

Where is the Windows port? and if there is one will it always be second rate compared to the Unix and Linux port?

Also where is the .Net driver?

Our take on RethinkDB vs. MongoDB

Related Stories