Things I wish I knew about MongoDB a year ago

http://snmaynard.com/2012/10/17/things-i-wish-i-knew-about-mongodb-a-year-ago/

by beastmcbeast • 14 years ago

316 108 14 years ago

nickzoic• 14 years ago

The count({condition}) one is a worry. I'm guessing it is slow in the case where it has to page the index in in order to count it. I wonder if it is still a problem where the index is used a lot anyway. A fix in MongoDB would seem a lot better solution than having everyone implement their own hacky count-caching solution.

EDIT: Actually, looking at the bug reports, sounds like maybe lock contention on the index?

The master/slave replication problem seems bad but I think it can be worked around (for my particular project) with a flag on the user session ... if they've performed a write in the last 30 seconds, set slaveOkay = false. Users who are just browsing may experience a slight delay in seeing new documents but users who are editing stuff will see their edits immediately.

tomschlick• 14 years ago

I'm so glad this wasn't another case of someone just ranting about using mongo for the wrong purpose and being mad about it a year later.

trafficlight• 14 years ago

I also appreciate how he pointed out positive things that he just wasn't aware of initially.

dia80• 14 years ago

Genuine question:

In what use cases does mongo kick mysql's ass?

I've used it a couple of times in hobby projects and enjoyed not maintaining a schema. I read so many of these 'gotcha' style articles and for example one commenter here wants to have a manual "recently dirty" flag to combat the master / slave lag mentioned in the article. I know it's faster (tm) but once you have to take in to account all this low level stuff you have to worry about yourself wouldn't it just be better to rent/buy another rack of mysql servers and not worry about it?

Look forward to learning something...

alexro• 14 years ago

IMO there is no rational explanation to this phenomena other than: people are different. Some get bored with stored procs and want same hassle but in another form.

morsch• 14 years ago

A cynical but insightful description of many kinds of progress and change. ;)

taligent4• 14 years ago

Yes. There is no rational explanation for people using the right tool for the right job.

Let me guess. You would build a skyscraper using a trough and cement in a bucket ?

diminoten2• 14 years ago

Mongo isn't so important to this question as ODB vs RDBMS. Here's some light reading:

http://en.wikipedia.org/wiki/Object-relational_impedance_mis...

MongoDB is just ODB, and MySQL is just RDB.

Besides, postgres is the real future!

dleblanc• 14 years ago

Not sure what Wu-Tang has to do with this, but....

Seriously though, is Mongo an ODB, or a document oriented database? ODBs/OODBs imply a much different use-case and functionality, and I think we ought not conflate the two.

taligent4• 14 years ago

PostgreSQL is the PAST and it will forever remain there until it solves its biggest problems.

It is the hardest out of all databases I've used to cluster, replicate, shard and manage. And the database world is moving towards scaling horizontally rather than vertically. I can push a button in say CouchDB to replicate and shard. Try doing that in PostgreSQL.

thibaut_barrere7• 14 years ago

MongoDB kicks ass in the following situations (real projects I did as a freelancer):

- dealing with semi-structured input (forms with some variability) and storing as a document, all while being able to query across the data

- used as a store to provide very flexible ETL jobs (with ability to upsert, filter/query, geonear etc)

For those situations, I would definitely use MongoDB again. As a RDBMS replacement, I wouldn't use it today.

jandy• 14 years ago

To elaborate on the semi-structured input point: Monogo and it's kin are great for EAV systems (http://en.wikipedia.org/wiki/Entity%E2%80%93attribute%E2%80%...), where your entities can have an arbitrary number of fields (often user defined). Trying to build this kind of system in a traditional RDBMs can be quite tricky.

dude_abides2• 14 years ago

To slightly rephrase the OP's question:

  In what use cases does mongo kick postgres's ass?

To the two points you mentioned:

- semi-structured input can be saved as hstore type or as json type;

- and for flexible jobs, you can use pretty much any popular language - PL/R, PL/Python, even PL/C if performance is really critical.

joevandyk• 14 years ago

fwiw, postgresql 9.2 supports a json datatype. you can efficiently access/query json fields using plv8 (http://code.google.com/p/plv8js/).

so you could have:

  create table form_results (                                                                                
    id serial primary key,                                                                            
    data json                                                                                         
  );

http://pgeu-plv8.herokuapp.com/ has more information.

olalonde• 14 years ago

For those like me who have no ides what ETL stands for: https://en.wikipedia.org/wiki/Extract,_transform,_load

mattparlane2• 14 years ago

My number one reason for choosing MongoDB is replication that just works out of the box and doesn't require either a read lock or shutting down the master to set up a new slave.

sausagefeet2• 14 years ago

As far as I can tell, Riak blows mongo away for this particular criteria.

jeltz3• 14 years ago

Serious question: What database requires read locks or shutting down the master for setting up a new slave? Is it MySQL?

Both sounds like really weird requirements for replication which defeats half the purpose of having it. PostgreSQL has never had any of these two problems. Still working on improving usability but with the addition and improvements of pg_basbackup I would say it is almost there. Hopefully 9.3 will get timeline switching to simplify failover.

untog• 14 years ago

I think there are legitimate reasons to use a "NoSQL" solution rather than MySQL. I'm more interested to know in what use cases Mongo kicks it's competitors asses? What are it's competitors, even? I'll admit that the NoSQL world is a slightly blurry mess to me, with different products seemingly optimised for different cases.

taligent4• 14 years ago

MongoDB is a JSON document store. Almost all other NoSQL are key/value type stores.

If you have a use case that centres around storing document style data then MongoDB will be better suited.

mrkurt2• 14 years ago

Eventually consistant replication is not unique to MongoDB, most DBs have an async replication option. Using "not Mongo" won't really solve it.

krichman• 14 years ago

An option is not a default, though, so using Postgres or MySQL absolutely would solve the problem of returning with success before persisting the data.

The contentious area isn't really that Mongo does this, it's because for whatever reason the people trying it don't expect Mongo to do this.

mrinterweb• 14 years ago

Map reduce across sharded servers comes to mind as an advantage. For that matter, horizontal scalability in general is a big advantage that many of the NoSQL data stores have over RDBMS databases.

elchief2• 14 years ago

When you have so many writes that sharding isn't enough.

When you make changes so fast you need a liquid schema.

When you want to make your boss learn map-reduce so he can query the data.

When the application can take care of integration and not the database itself.

lwat• 14 years ago

> When you have so many writes that sharding isn't enough.

Does Mongo still have a global write lock?

leif10• 14 years ago

This will be obvious, but I work at Tokutek.

> When you have so many writes that sharding isn't enough.

TokuDB does indexed insertions very fast. [1] We've even plugged ourselves in underneath MongoDB (just for fun) and we beat them too. [2]

> When you make changes so fast you need a liquid schema.

TokuDB supports lots of schema changes with zero downtime. [3]

> When you want to make your boss learn map-reduce so he can query the data.

Can't help you there, but I can make you not need to torture your boss that way.

> When the application can care of integration and not the database.

What if it didn't need to?

We also get fantastic compression [4], retain full transactional semantics, and lots of other fun stuff.

Email us if you're curious!

[1] http://www.tokutek.com/resources/benchmark-results/benchmark...

[2] http://www.tokutek.com/2012/08/10x-insertion-performance-inc...

[3] http://www.tokutek.com/resources/benchmark-results/benchmark...

[4] http://www.tokutek.com/resources/benchmark-results/benchmark...

jeremyjh• 14 years ago

One issue with MySQL in large databases is that schema changes are extremely expensive, so much so that you'll be making design decisions around it (e.g. how do we implement this feature without executing our two-day alter table statement). Not all RDBMS have this problem to such a degree but none can escape it entirely.

A lot of the gotchas that he notes are related to design trade-offs with different default behavior than an RDBMS typically would have. For example as your system gets large enough in MySQL you may find you have to do asynchronous replication as well, and then you will have similar problems with dirty reads.

bsg752• 14 years ago

> e.g. how do we implement this feature without executing our two-day alter table statement?

With MySQL being so prevalent, IMO this is the principal reason why the concept that RDBMS' are hard to work with exists. None of the other major platforms have this issue, but if a developer's only exposure is to MySQL, then the idea of schema alters are always fearsome.

This is not to say that ALTER TABLE statements are always quick - if constraint checks or default values are included, there can be long runtimes in PostgreSQL, MSSQL, Oracle, etc. - but significant downtime to add an empty nullable column is just plain stupid for a platform that has been around as long as MySQL.

MySQL has forced a major population segment of developers to toss the advantages of DB side validation and rich query languages for the purpose of _avoiding_MySQL_. Sure the whole object-relational impedance mismatch exists, SQL is hard, yada-yada, but I have never seen these reasons cause as much angst as taking a service offline to add a column to a table.

10gen should have a picture of Monty in their CFO's office.

lars512• 14 years ago

The inconsistent reads in replica sets is something we've come across with MySQL read slaves as well. I think it's a gotcha of that whole model of replication, rather than a MongoDB-specific issue.

jbert• 14 years ago

One way to resolve it is to mark that user or session (or even just request) "sticky to the master" for long enough to cover your normal replication delay.

When we saw it before, ensuring that a given request which issued a write also read from the master was sufficient. (sub-second replication delay).

mgummelt6• 14 years ago

This may help in the majority of the cases, but many applications also can't tolerate inconsistent reads across users/sessions.

mgummelt6• 14 years ago

I'm not aware of any database that solves this problem. Is there one? As far as I know, mysql reads must be distributed to the slaves at the application level, which has no knowledge of master/slave inconsistency. I suppose the time delta between master and slave can be queried, but that still doesn't protect from race conditions/inconsistent reads. This is actually why we chose to only utilize slaves for data redundancy rather than read throughput at my last company. Inconsistent reads weren't tolerable.

orthecreedence• 14 years ago

Riak does. You say, when writing, "please don't return until this data is replicated on 2 servers." And when reading, "please only return a successful read if this data is read from 2 servers."

So you have R = 2, W = 2, R+W = 4, and if your replication (N) val is 3, you're fine (you're always going to get consistency if R+W > N).

Riak is cool.

nerfhammer• 14 years ago

This is called semi-synchronous replication

http://dev.mysql.com/doc/refman/5.5/en/replication-semisync....

arielweisberg2• 14 years ago

Shameless plug (hey Tokutek is doing it), in VoltDB replication is synchronous so it doesn't have this problem.

Latency in the current version is nothing to write home about, but in V3 latency with replication is 600-1000 microseconds. Group commit to disk is every 1-2 milliseconds.

V3 also allows reads to be load balanced across replicas and masters so you gain some additional read capacity from replication. V3 also routes transactions directly to the node with the data so you don't use capacity forwarding transactions inside the cluster.

You get to keep transactions to. Now go figure out what you don't get to keep ;-)

bsg752• 14 years ago

Any platform that supports synchronous replication?

http://www.postgresql.org/docs/9.1/static/warm-standby.html#...

nevinera2• 14 years ago

>Range queries are indexed differently

If I'm reading your description right, this is hardly mongo-specific. Try it in mysql, for example:

(index is [:last, :first])

  select first from names 
  where last in ('gordon','holmes','watson')
  order by first;

An index is an ordering by which a search may be performed - to illustrate, the index for my small table looks pretty much like this:

  gordon, jeff
  holmes, mycroft
  holmes, sherlock
  watson, john

Unless the first key is restricted to a single value, it can't order by the second key without performing at least a merge-sort. They aren't in that order in the index.

foobar2k• 14 years ago

He never said it was mongo specific

nevinera2• 14 years ago

>Things I wish I knew about MongoDB a year ago

The post reads as a series of criticisms about mongo. I don't love mongo, but I'm not aware of any data store that can perform that type of query purely from an index.

Now, the description was vague enough that he could have been describing a real bug I'm not aware of - at one point I've seen MySQL decide to use an index for sorting instead of for filtering when that query plan was 500x slower. If mongo has a bug like that one, disregard my comment please. :-)

chris123• 14 years ago

Is MongoDB more marketing hype than quality product? I've heard it before and this article seems to point in that direction as well.

lmm2• 14 years ago

Yes, but only because it has ~infinite marketing hype.

It isn't and shouldn't be a general replacement for a RDBMS; it makes some interesting sacrifices for performance that you have to understand before using it. But it is very much a quality product; it makes some easy things very easy and some very hard things possible.

kokey• 14 years ago

I think it's generally full of gotchas similar to that of SQL databases like MySQL and Oracle. In fact, most of the issues mentioned in this article, like delayed replication, indexed queries and using 'explain' are issues I've had to deal with in MySQL and Oracle. Most of these databases are fine out of the box for small scale use, but when you scale up you have to deal with these 'gotchas' like indexing, partitioning, bulk loading, and having to profile everything etc.

bengaoir• 14 years ago

I wish I knew that it sucked.

jameswyse• 14 years ago

One thing I love MongoDB for is it's geospatial indexing abilities: http://www.mongodb.org/display/DOCS/Geospatial+Indexing

Was a really nice surprise when I was building a location based web app.

wakaflaka• 14 years ago

jsemrau• 14 years ago

That was our use-case as well. And it works fine for this but just in the application layer. We are not using Mongo for data storage (at least we are not trusting it to hold it for long)

Things I wish I knew about MongoDB a year ago

Related Stories