The growing irrelevance of MongoDB

http://www.itexto.com.br/devkico/en/?p=60

by jraedisch • 12 years ago

92 112 12 years ago

itexto.com.br

jraedisch4• 12 years ago

If transactions are the biggest problems, there are driver solutions like http://godoc.org/labix.org/v2/mgo/txn for that. I am not sure, how they compare to "real" transactions. Still missing "joining" in some cases though. Otherwise I am ok with MongoDB so far.

stephen123• 12 years ago

If another host connects to the db, and the transaction doesn't span to that second host. Its not as safe as sql transactions.

niemeyer2• 12 years ago

The transaction does span across hosts with the linked package.

politician• 12 years ago

Jeez. Please do not mention client-library-managed transactions and then put "real" transactions in scare quotes. At least read the Wikipedia article first.

jraedisch4• 12 years ago

I just wanted to distinguish between the two. I wasn't scared in any way but read up on scare quotes now - so thanks for that. http://en.wikipedia.org/wiki/Scare_quotes

StavrosK• 12 years ago

I didn't read it like that at all, I interpreted it as the gp distinguishing between the two.

niemeyer2• 12 years ago

There are further details about how it works in this blog post:

http://blog.labix.org/2012/08/22/multi-doc-transactions-for-...

While it does work, it's still a workaround for the lack of first-class transactions in MongoDB. It offers a more limited API and requires care on the developer side.

Despite being the author, I do hope it gets obsoleted by more convenient first-class transaction support in MongoDB itself at some point in the near future.

crzrcn• 12 years ago

The submission url is malformed :(.

tech-no-logical• 12 years ago

I think it should be

http://www.itexto.com.br/devkico/en/?p=60

jnaour• 12 years ago

Probably this link: http://www.itexto.com.br/devkico/en/?p=60

scarygliders• 12 years ago

Site seems to be overloaded or broken in some other way.

Google cache : http://webcache.googleusercontent.com/search?q=cache:wwwdesi...

collyw6• 12 years ago

I have seen many people try and shoehorn various problems into MongoDB, when in most cases a relational database would have been better suited.

I have yet to see a real use case for Mongo unless you are building a Facebook clone. Can someone suggest when it is actually useful over a properly tuned relational database?

I guess I kind of reached the irrelevance stage just thinking about the sort of problems it would be suited to rather than actually building anything.

Harrisburg3• 12 years ago

Think of an Amazon like product catalogue.

I don't want to create a schema in a relational database for that kind of use case. A MongoDB fits as well.

Edit: Sorry for my short and unprecise comment. I regret posting it, loosing all my Karma. Yes, of course I would use a schema, but I may use hundreds to get my products presented with all attributes and variants available. There is a set of attributes that will remain for all products, others will vary, some will change and will be dropped. Some may be conditional, depending on size or color. I could model that in RDBMS as well, adding those properties in an additional JSON or something else.

Yes, I'm also mainly using relational databases and just wanted to give an example of a use-case for MongoDB. I regret, sorry.

rjaco31• 12 years ago

Care to elaborate? Beside maybe for the suggestion system, I really don't see what's the gain compared to a relational database.

CmonDev• 12 years ago

You can always just stick JSON into a separate table for those cases. So that alone is not an argument.

collyw6• 12 years ago

I wouldn't personally say it fits better. And you are throwing away a ton of useful features that come with a relational database.

crdoconnor3• 12 years ago

So what happens when you want to list all of the products in a category?

ris3• 12 years ago

"I don't want to create a schema..."

You are always going to end up creating a schema, whether it's explicit in your tables or implicit within your code. Otherwise you end up with completely heterogeneous data which is impossible to query in any useful way.

This is a red herring.

rimantas• 12 years ago

I'd say elasticsearch is often used for this kind of tasks.

annnnd7• 12 years ago

Lately (and not so lately ;) there has been a lot of bad press about MongoDB. We use it extensively as part of our product (on single servers) and it fills this role nicely. It has a few drawbacks, most notably huge disk space requirements (MongoDB has no compression) which we are hoping to solve with TokuMX (haven't tried it yet). It has some other quirks too, but in general it just... works. And I love using a document DB because it lets you use relations just when they are needed. With relational DBs you often have to use complex joins even for things which should be incredibly simple. On the other hand, I do miss JOINs when I need them (though we solved that on app level). And I would appreciate a way to define schema... :) But would I exchange document DB for relational DB for that? No (for the current project).

Maybe I'm getting old, but I really want DB to just work (unlike HBase, at least a few years ago). Other than that, I'm flexible.

collyw6• 12 years ago

Postgres and MySQL just work as far as I am concerned.

andrea_s4• 12 years ago

The good news is, you can just wait a few weeks and get the possibility to use WiredTiger inside the upcoming 2.8.0. That will solve a few pain points that we've been dragging along for years (namely, document-level locking, disk requirements, multi-document transactions).

ttty• 12 years ago

And I would appreciate a way to define schema

Use mongoose!

On the other hand, I do miss JOINs when I need them (though we solved that on app level).

How you do that? (maybe is different than me)

What about transactions?

JulianMorrison• 12 years ago

I could see its "capped collections" being useful for log analysis.

nextw33k• 12 years ago

Can someone suggest when it is actually useful over a properly tuned relational database?

In your own question you kind of hint at it. You need a tuned database. NoSQL lowers the barrier to entry for fast persistent data storage with replication. NoSQL doesn't replace SQL databases, it's looking to optimise a use case where transactions are not required.

Personally I use Mongo a bit like a cache, sitting in front of a SQL DB. Things that need to be ACID are handed back and forth. Those that don't are kept in Mongo and I get the best of both worlds.

ddebernardy• 12 years ago

NoSQL doesn't replace SQL databases, it's looking to optimise a use case where transactions are not required.

I'd be curious to know in what contexts anyone would not want transactions.

I readily see the case for memcached or equivalent when it comes to caching -- it is very valid, and indeed useful, when slightly outdated data is perfectly acceptable. I can even picture how you might be using MongoDB to do the same, even if you admittedly have me wondering how you're invalidating the mess.

For a persistent data store, however, I'm honestly at a loss. In my own experience, transactions are needed as soon as there's a remote possibility that a concurrent write may occur. Even embedded systems need them, when you're threading statements concurrently for performance reasons, or when you're subsequently merging local data with another node. (See CoreData/iCloud bugs for what occurs when you ignore ACID in the latter case.)

andrea_s4• 12 years ago

I'd say a graph database would work much better for building a Facebook clone compared to a document store like MongoDB... And a relational database would still be better than the document store.

I've been using Mongo happily to provide online analytics solutions, but the advantage is mostly from the development side, not really from the performance side.

On the other hand, this kind of approach is great for attribute matching, which is usually a nightmare to do properly with a RDBMS.

phpnode4• 12 years ago

I have yet to see a real use case for Mongo unless you are building a Facebook clone

People seem to keep making the mistake of building a social network in mongo. It's entirely unsuitable for that.

Source: I've helped 3 companies move their social networks from MongoDB to OrientDB when they figured out that MongoDB prevents them from shipping features that users expect, e.g. friend-of-a-friend style queries

filearts3• 12 years ago

What has your experience been like in deploying and working with OrientDB? This is a bit of really cool tech that I've been keeping an eye on for some time but haven't gotten around to really playing with it.

What server language were you using and where were you deploying?

proveanegative• 12 years ago

Did they choose OrientDB instead of Neo4j for the license or for technical reasons?

Kiro2• 12 years ago

I just like being able to insert a random JSON into a collection and query it by any of its properties. Not sure how I would do that with a relational database.

adamors• 12 years ago

PostgreSQL has full support for JSON [1]. So, quite easily in fact.

[1] http://www.postgresql.org/docs/9.4/static/datatype-json.html

thorin• 12 years ago

You can do this in Oracle 12 and I think in Postgres. You might want to think about why you need to store unstructured data as part of your application though and query it. If it's expected to live a long time and be queried it should probably be structured. If not you could store it (a serialized object) as a blob if you are just caching some state.

https://docs.oracle.com/database/121/ADXDB/json.htm#ADXDB624...

TheDong• 12 years ago

Facebook clone

Diaspora is a social network that made the mistake of choosing mongodb for their first iteration. Here's a blog post where one of the involved developers explains why it's not good for that, even: http://www.sarahmei.com/blog/2013/11/11/why-you-should-never...

andrea_s4• 12 years ago

To be fair, this is a poor technical choice that no reputable DBA should ever make... Not really the technology's fault.

Arnt4• 12 years ago

I've used it and been happy. If you want something that's simple to use, know your schema partly but not entirely, and you don't mind losing a few writes, then mongo fits.

Relational databases place such emphasis on reliability. Mongo is generally reliable but only two or three nines, and it's simple.

For example, if you're collecting sub-cent line items for invoices, you might prefer to collect 99.x% of the line items simply over 100% at greater complication and expense, particularly if you can measure x.

gasping3• 12 years ago

What? When would it ever be OK to lose parts of an invoice. Mongo apologists baffle me sometimes. To anyone considering MongoDB, don't be fooled, you probably want to stick with an RDBMS.

mietek• 12 years ago

This is satire, right? Right?…

nevi-me• 12 years ago

Hopefully a useful example, would love some comment.

I've been building https://rwt.to and https://movingjoburg.co.za with storage backed by MongoDB. Both are transit apps/websites in South Africa, could be more useful, except that we spend most of our time collecting and capturing data.

At the time I was very inexperienced (I guess I'm a bit better now?), but these are the reasons why I chose MongoDB.

1. Geospatial query support out of the box (this is during 2.0-2.2). It's now great with GeoJSON, I store routes and stations as GeoJSON, and querying them is very easy. Compare with PostgreSQL + PostGIS.

2. Some bus services have weird schedule structures, and I needed to be able to generate those schedules, I store schedules as a giant JSON file, and I can query that file to find when the next bus/train is. It's now possible with JSONB in PostgreSQL, but trying to fit that into a relational model wasn't worth the pain at the time, especially when there's always edge cases that I have to cater for at times.

3. To create a pricing engine on rwt.to, I needed a very flexible schema, because I have to cater for many cases (are there bus/train transfers, discounts, and quite different rules for those transfers and discounts). MongoDB gave me the ability to vary my schema in the instances where I needed to store different types of data. I'd rather contend with that than have a table with 100 columns to do the same. To head off on a tangent, in GTFS this data (fares) is computed and stored in the table, but I calculate fares at query-time because there's sometimes a lot of rules to consider, making calculating them and caching them like GTFS unpleasant.

4. Other reasons was because it was pleasant to work with MongoDB + Mongoose.js. This is very important as I'm the sole developer, and due to my line of work, I don't get all the time I need to work on both projects. For my core transit data, I won't reach >20GB even when I manage to collect all the data for South Africa, so I won't have to contend with the 'big data' issues that other users would face.

PS: Out of interest, if anyone would like to try rwt.to to see how it works, here are some example links: - https://rwt.to/*mrwF8w0u - https://rwt.to/*mrCLEZaE - https://rwt.to/*mrOkEDKH

lmm2• 12 years ago

I have yet to see a real use case for Mongo unless you are building a Facebook clone. Can someone suggest when it is actually useful over a properly tuned relational database?

Seamless auto-sharding, supported as a first-class use case. Particularly in a cloud environment where you can have your system automatically spin up more hosts as load increases (and take them down as load drops) with zero human intervention, which is really nice.

There are plenty of alternatives in that space nowadays (more than when mongo first came out), and reasonable people can disagree over whether to use mongo rather than e.g. cassandra, riak, or shonky third-party clustering addons for mysql/postgresql, but I've yet to see an affordable relational database with out-of-the-box auto-sharding support that comes anywhere close to what mongo offers.

(Also, first-class support for async-io clients. This is about client libraries rather than the server itself, and purely an artefact of when mongo was released, but if you're calling e.g. PostgreSQL from the JVM, most of the libraries are oriented towards blocking JDBC which is "good enough" (there are valiant efforts like https://github.com/mauricio/postgresql-async but you can't use them with established higher-level libraries). Whereas every mongo library offers a callback- or future-based API, and the higher-level libraries are built around this)

girvo• 12 years ago

but I've yet to see an affordable relational database with out-of-the-box auto-sharding support that comes anywhere close to what mongo offers.

FoundationDB is a bit weird, but it handles automatic scaling-out, and has an ANSI SQL interface (backed onto a K-V store): https://foundationdb.com/key-value-store/white-papers/scalab...

Caveat: I've not used it for more than hobby stuff yet, but it's a fascinating bit of technology!

deepGem• 12 years ago

MongoDB is not at all good for Social networks or anything resembling a graph. You are better off with Titan (horizontal scaling) or Neo4j (vertical scaling). Neo4j offers this great ability to query by a path, which no other database offers.

enobrev• 12 years ago

I generally agree in regards to shoehorning data into MongoDB. I don't agree that a facebook clone fits appropriately. When thinking through a project, my mental model generally self-configures into third-normal form, which is to say I literally think in the manner of a relational database.

That said, I have found one specific use-case where it has incredibly worked well, which is as a JSON-based file system in which I can query into the document structure to find the records and data I need en-masse with very simple queries. A project I've been working on for a few years has its own custom schema that fits far better into a JSON hierarchy than a relational one. In this specific system there are very few records (compared to our primary RDBMS which has millions), and when not being explicitly maintained, the files are generally static. The single-collection database in question with a few hundred JSON objects would likely have to be split into at least 10 tables with thousands of records each.

When it comes to using said "files" in production, I index the objects into memory for quick retrieval. In order to populate said cache, MongoDB allows me to query into the JSON file structure and retrieve the necessary "files" very simply into however many indices I need. And that's where it has shined in my experience; Not as a live read-write datastore for production use, but as read-heavy storage for a relatively small amount of JSON data with a deep hierarchy that would require an annoying amount of joins and subqueries for otherwise simple queries.

I've also tried it out with our multi-system logging system using capped collections (JSON formatted messages transported and collected with rsyslog). But we grew beyond its limits almost immediately and ended up in far more manageable territory with kibana / elasticsearch.

pjmlp2• 12 years ago

Can someone suggest when it is actually useful over a properly tuned relational database?

Maybe when developers don't want to bother to learn SQL?

On my CS degree we got proper introduction to SQL and the relational algebra behind it, so for me it is just a tool, not a scary monster.

I really love the data integrity options I have at my disposal with relational databases.

So for me this all NoSQL fad never made much sense. Then again, I never had to deal with a Facebook like scale problem, only daily reports from mobile network operators across their whole network elements.

octix2• 12 years ago

It's not about Facebook clone, they still use MySQL for some functionality (i believe), but not as RDBMS but more like NoSQL...

NoSQL means model your data in a different way, the way you'll be using it. For some it makes more sense, for some may not.

For example, compare implementation of a tagging system in mongo and any other RDBMS, which one is easier?

khalidmbajwa• 12 years ago

We are building a music streaming website. Think of it as a Spotify for Pakistan and Mongo has been wonderful for the kind of Data Structure we have. Its perfect for a catalogue like structure where objects can very easily nest inside each other. Each Artist has multiple albums, and multiple artists have multiple songs. All of this nestles very neatly inside eachother, and when i want a song, i almost always need its album and artist data, so i can very easily get that. Mongo is blazing fast on such kind of nested data structures. The only slight problems is where you have playlists and need to reference songs for a certain Artist. That could be countered by Data Duplication, since the song artist data etc once entered will very rarely change. So for us it makes perfect sense. You really need to understand what your data model is, and what you want from it, and then Mongo will be your friend, other than that it will cause you a world of pain.

fragmede• 12 years ago

How does that hierarchical model work when songs or albums have more than one artist?

Why does a traditional relational db fail for such a well-defined data model? I mean, there's nothing about 'given an song (id), retrieve artist and album' that would make mongo inherently better.

jbergens3• 12 years ago

For some systems the development time is much shorter with a document db (or a graph db) than with a relational db. That can be more important than pure performance. Basic scalability is also usually better since it often easy to shard data across servers.

avasdvasdv3• 12 years ago

I have seen many people try and shoehorn various problems into MongoDB

MongoDB is easy and was used by web "developers" too lazy to learn SQL and RDBMs.

MongoDB is fine if you don't really care about the data. If you don't need ACID compliance and can afford to lose transactions.

collyw6• 11 years ago

/dev/null would be faster in those cases

Harrisburg3• 12 years ago

I can't agree. MongoDB 2.8 with WiredTiger is a performant NoSQL solution.

The way you solve things in document dbs like MongoDB is the key - not the fact that you get competitve performance in RDBMS as well.

I like Postgres and the possibility to easily use JSON data. But I still prefer to build applications the NoSQL way...

bigiain• 12 years ago

MongoDB 2.8 (in RC4 right now, due out "in early January" last I heard) has the WiredTiger storage engine, and as well as high performance and compression also gets document level locking and multi document transactions. The roadmap says this'll become the default storage engine in 3.0 (3rd quarter 2015). I think the blog author's claim that for basically those omissions "MongoDB can still beat TokuMX on a future release. But only in a future release. Today it can’t." observation is only true if you ignore the development releases completely, and will be incorrect within a week or two.

Sure, if you want SQL or traditional relational database, a traditional relational SQL database is a better choice. Using Postgres or Oracle for workloads better suited to tied hashes or BerkleyDB doesn't automatically become "right" though.

jraedisch4• 12 years ago

Could you provide a source for "multi document transactions" support in 2.8? According to this comment they are supported by WiredTiger but won't be in the MongoDB API anytime soon: http://blog.mongodb.org/post/102461818738/announcing-mongodb...

dm_mongodb• 12 years ago

you are correct, that's not in 2.8. might want to follow this jira: https://jira.mongodb.org/browse/SERVER-11500

that said it will be a big release in other aspects, with pluggable storage engines now, and WiredTiger specifically.

nchelluri• 12 years ago

Unfortunately I didn't find much information in the article.

Basically I got out of it:

- he likes JS, and was comfortable with MEAN-stack (MongoDB/Express web framework/AngularJS/Node.js)

- he found that for document oriented purposes MongoDB could sometimes be a nice fit

- he found that replacing MongoDB with the drop-in replacement TokuMx (seems like a MySQL/MariaDB type of idea), he could get a big performance increase

- he found that with Postgres 9.2+ using the JSON/document storage there he could get some of the relational benefits and some of the document benefits, and ACID

- he likes ACID because of transactions/handling multiple documents at once

Ok, so I guess I was wrong, there is a bit there. But: this is all stuff that I've heard several times about MongoDB.

One of the things he got into was that he learned when MongoDB was appropriate and when it wasn't -- but IMO he didn't really go into detail here other than to say: joins and transactions. I wish there was more substance than that.

I also seem to recall reading recently that even supposedly ACID compliant databases have problems with transactions under load and that the only way to really get full ACID compliance is to treat transactions as serializable - perhaps I am wrong but I think this is the gist of what I've learned.

Personally, I'm working on a project right now that uses MongoDB and I definitely do miss joins, transactions, and schemas. I am not even remotely sure where one would benefit from a system that had documents with arbitrary fields in it. (I didn't pick the technologies for my project.)

I would really like to know when to choose which database - I feel like I know the bare basics of several, and none of them in depth. I really don't have time to learn them all. I feel like even when I'm building one application it's only some months after deployment that if I'm lucky or made a mistake that I will run into some actual performance constraint.

crdoconnor3• 12 years ago

I would really like to know when to choose which database

I have asked this question before - under what situations is Mongo not just equally good as Postgres - but actually better?

The only really coherent answer was that it was easier to configure replication (presumably because it chooses a lot of defaults for you). I'm not even sure that was a good thing given the number of obscure bugs that can arise from incorrectly configured replication.

Postgres even seems to be more performant at NoSQL use cases (using the JSON store) than Mongo, which is frankly embarrassing.

gasping3• 12 years ago

MongoDB will always be a relevant example of false advertising and over-marketing (to the point of 10gen probably opening themselves to litigation) and how we all need to stop drinking the kool-aid.

There is LITERALLY no reason to use MongoDB today. If you're thinking of using MongoDB, for the love of god just try PostgreSQL.

mwj2• 12 years ago

this is a ridiculous comment.

we use a mongodb cluster for a multi-hundred GB document store that powers ~50 various worker instances, 2 sites and an API with sub 100ms response times. we have never lost data or experienced downtime worse than other dbs i've used in the past.

at the end of the day you just need to do your job properly and read the documentation, not blame the tool when you can't use it properly.

*disclaimer - this is not to say postgres isn't awesome.

kicolobo2• 12 years ago

Actually, they posted thw wrong link. That article is not written by wwwdesigned, but by me. Here is the original text: http://www.itexto.com.br/devkico/en/?p=60

jraedisch4• 12 years ago

I actually came there via two other aggregators (Twitter and Prismatic) and thought that I had found the source. Will be more careful next time - sorry about that. In any case it was an interesting read.

kicolobo2• 12 years ago

Thanks!

dang• 12 years ago

We've changed the URL from http://wwwdesigned.com/growing-irrelevance-mongodb/. Did that site publish your article without permission?

anilmujagic• 12 years ago

I never understood the hype around MongoDB. Maybe it is the fact that I started programming when SQL DBs were the prevalent/mainstream option, and got too much used to it... Anyways, Im glad people are recognizing the hype and realizing that RDBMS has enough to offer.

octix2• 12 years ago

It may be hype, but when MongoDB was released, what other RDBMS was offering the same functionality? I see Posgresql + JSON mentioned, but when was JSON support added?

I personally like MongoDB for: - flexible schema (less migration pain) - easy tags implementation - product attributes (list of name-value pairs) - GridFS - store binary files - nested documents for analytics (ex: a record for each day with a nested doc for each hour)

I wish MongoDB had - some support of join - multi-master replication

I personally used MongoDB as primary storage (yeay dot me), but currently prefer it as secondary storage.

As any product it evolves and I expect to see more improvements. I think MongoDB brought NoSQL to the masses :)

Just my 0.02. Thanks.

snird• 12 years ago

Everything you said is true, but I don't think the argument here is about the past, but rather about the future.

MongoDB was great and innovative compared to other solutions when it started, but now, it's dragged behind while other solutions are much better.

jbergens3• 12 years ago

Then we should probably compare other NOSQL solutions to relational databases. Like CouchDb, CouchBase, FoundationDb, Neo4J, etc.

FallDead• 12 years ago

Nice writeup, I always felt mongodb as a really fast product prototyping database . After you get to certain point you in web scale I would guess you would use one of these alternate solutions.

The growing irrelevance of MongoDB

Related Stories