MongoDB Performance & Durability (2010)

http://redbeard0531.s3.amazonaws.com/mikeals_blog_backup/MongoDB+Performance+&+Durability.html

by wisty • 15 years ago

9 11 15 years ago

redbeard0531.s3.amazonaws.com

wisty3• 15 years ago

I'm just starting MongoDB, and I really like it. It's got nice documentation, and a nice culture.

But it does have weaknesses, and having just been bitten hard by one.

I had a schema something like "db.post.insert({'tags'=['python','mongodb']}).

One tag just couldn't be reliably deleted (via "$pull:{'tags':'immortal tag that will not die'}.

It turns out, I had made "tags" unique, but somehow managed to get a duplicate tag in. (Before the unique index? After? I forget). Trying to delete the tag would "throw" a E11000 error, but since mongodb doesn't check errors it would just result in this immortal tag sitting there. Smirking.

Apparently, using a unique index on an array is the wrong thing to do anyway (does it mean different posts can't share tags? or that you can't have 2 posts with exactly the same set of tags? I'm not sure), and you are just meant to use $addToSet. I'm not sure how you are meant to clean an existing array with duplicate elements though. Perhaps you just have to check in the app layer, then update: db.posts.update({'_id':post['_id']},{'tags':set[post['tags']}.

Lesson 1: if something is failing a lot, or might fail in an annoying way, use db.error() (in Python ... or whatever the last_error command is in your client).

Lesson 2: don't try to use unique indexes on array elements.

Otherwise, it's a very nice db.

jwilliams• 15 years ago

Keep in mind this is a year old and MongoDB has since implemented journalling (although afaik it's not on by default yet).

It's not specifically a response to this article - But you can read 10gen's view on durability here: http://blog.mongodb.org/post/381927266/what-about-durability

wisty3• 15 years ago

As I would put it:

In the event of a single server crash, Mongo may need to be restored from backup, or another master. CouchDB handles this much better, and can just restart where it left off. But in the worst possible single-server scenario, there's smoke coming out of your server and the hard drive is toast. CouchDB and MongoDB perform about the same - if you had replication or a recent backup, you are fine. If you didn't, you are f*cked.

CouchDB is better for small servers which crash a lot and can't use replication. You can run it on a mobile phone. Mongo is better for logging (where you need fast writes, and don't care about a couple of lost records). Most uses will be somewhere in between, and you need to know the limitations of the stack, so you can work around them.

rit2• 15 years ago

If you are running with journaling enabled, you should get a much stronger crash recovery case.

In the event of a crash, the journal will be replayed: http://www.mongodb.org/display/DOCS/Journaling

rdtsc4• 15 years ago

One needs an off-site backup of the data anyway to handle the 'smoking server' case anyway. But I don't see how saying that a server can catch on fire so we should bother with default durability guarantees anyway is a sane choice. It is like saying that cars can catch on fire so why bother wearing seat-belts.

rdtsc4• 15 years ago

From the article:

> To start, there are some very practical reasons why we think single server durability is overvalued.

Most people should just stop reading at that line.

> If you have durability through a transaction log, then you have to replay it to come back up.

That shows a serious misunderstanding how append-only can work. In CouchDB's case there is no need to replay the whole file just find last consistent piece of data at the end. So seek close to end & find last consistent header.

> If you have a master and slave in the same data center and you lose power, both will have to recover which could take 5-30 minutes.[1]

If you use MongoDB your DB apparently can get corrupted and you won't find out about for another month (link from article):

http://www.korokithakis.net/posts/using-mongodb-for-great-sc...

> If you have a 100 node cluster, worrying about every machine is a liability.

I see having to buy 100 nodes to get durability as a non-starter.

I mentioned this before and I'll say it again that calling their product a database and by default not returning write results is dishonest.

10gen employee (Kristina Chodorow) replying to the article admits that "We did this to make MongoDB look good in stupid benchmarks"

Well that's fine but then just call it rrdtool-ng

wisty3• 15 years ago

I think she was being sarcastic. But yes, it does make MongoDB look good (as in, better than Redis, not just 10X the speed of CouchDB) in stupid benchmarks.

MongoDB is designed to be fast by default, which I like. But it can bite you.

I like its speed, and its interface, binary data (which saves disk space), and documentation. I'm not sure if it's data structure is sound though. I'd really like something with CouchDB's append only log, and MongoDB's interface (fast and easy), rich updates, and documentation.

mikealrogers• 15 years ago

I wrote this, quite a while ago.

MongoDB implemented an append only log which is not enabled by default.

The rest of the article is still accurate, and the fact that durability isn't on by default IMO continues to prove that my characterization of their attitude towards durability is still accurate.

MongoDB Performance & Durability (2010)

Related Stories