Why The Clock is Ticking for MongoDB

http://rhaas.blogspot.com/2014/04/why-clock-is-ticking-for-mongodb.html

by turrini • 12 years ago

92 136 12 years ago

rhaas.blogspot.com

dkhenry• 12 years ago

This is clearly a flamebait title. The article doesn't say anything about why MongoDB is running out of time other then "PostgreSQL is making real progress as a document store". I think unwittingly the author has identified why MongoDB is not running out of time. It still has a huge lead on RDBMS in a very common and useful workload. If they can continue to make progress while other engines catch up with some of the document oriented features ( as Postgre has done ) then they will still have compelling features to offer. If they do nothing while other engines make progress then of course they will fail.

If anything all this points out is that Document stores, like MongoDB, have a real market where they excel and other engines are playing catchup.

simonw2• 12 years ago

The flame bait title was in response to a similar title on zdnet: "MongoDB chief: Why the clock's ticking for relational databases"

The author should probably have quoted that title rather than expecting people to follow the link.

jeffdavis4• 12 years ago

"It still has a huge lead on RDBMS in a very common and useful workload."

There's very little barrier to entry there, though. With 9.4, PostgreSQL will work better on a lot of JSON workloads than MongoDB. And postgres will continue filling in features to cover essentially all of the use cases MongoDB offers now.

MongoDB will have a much harder time supporting things like joins (and normalization which reduces redundancy), new data types, good (and flexible) transaction semantics, sophisticated indexing strategies, cost-based optimizer, useful constraints, etc.

This is one reason why XML databases never went anywhere -- existing SQL database systems just said "we can do that, too" and the XML-only systems were relegated to a small niche. Unless MongoDB really continues to innovate, it will have the same fate. JSON+Sharding is a good combination, but it's just not enough to be a lasting player in my opinion.

pilif5• 12 years ago

I really don't see how a fixed schema is seen as such a bad thing by many NoSQL advocates. In most databases, altering a schema is an operation that's over quickly and in many databases it's easily reversible by just rolling back a transaction.

The advantages of a fixed schema are similar to the advantages of a static type system in programming: You get a lot of error checking for free, so you can be sure that whenever you read the data, it'll be in a valid and immediately useful format.

Just because your database is bad at altering the schema (for example by taking an exclusive lock on a table) or validating input data (like turning invalid dates into '0000-00-00' and then later returning that) doesn't mean that this is something you need to abolish the schema to solve.

Just pick a better database.

rsynnott• 12 years ago

In most databases, altering a schema is an operation that's over quickly

I think many people were burned by MySQL InnoDB, which modifies (or modified; I'm still on quite an old version) schemas by making a new table and copying the data, and don't realise that all databases don't do that.

emeraldd• 12 years ago

Was that InnoDB or MyISAM? I didn't realize it used the same technique on both.

neverminder2• 12 years ago

I can confirm that is still the case with latest MySQL version, which is why my company is moving to PostgreSQL.

lucian19002• 12 years ago

It's even better: with an explicit schema you can trivially do data migrations!

With something without a schema, data migrations are almost impossible to do correctly.

danpalmer2• 12 years ago

I think trivially is over simplifying it, but certainly migrations are much easier with a schema.

threeseed12• 12 years ago

Data migrations are rarely just about adding or removing a few columns. It also involves data manipulation. Schema doesn't help you with this. So at some point you have to write data migration code. In which case (and if you are using an ORM) why not do the schema change at the same time.

And sorry but it is complete nonsense to suggest that data migrations are almost impossible on schema less systems. I've done countless myself and people are doing it every day.

gnufied2• 12 years ago

I do not disagree with your primary PoV that schemas can be a good thing, but:

a. Schema changes are considered DDL statements and hence can't be simply rolled back via transaction in most databases (as far as I know).

b. If you have millions or more rows and you need to introduce a schema change, it can get painful to do so and can result in app downtime. It is a well documented problem with some well documented work arounds.

jkodroff• 12 years ago

I agree with your points, but PostgreSQL does support transactional DDL: http://wiki.postgresql.org/wiki/Transactional_DDL_in_Postgre...

pilif5• 12 years ago

a. Schema changes are considered DDL statements and hence can't be simply rolled back via transaction in most databases (as far as I know).

In the database of my choice (Postgres. Since 7.1), DDL is transactional and can be rolled back.

b. If you have millions or more rows and you need to introduce a schema change, it can get painful to do so and can result in app downtime.

In many databases and for many schema changes the workarounds are trivial (like adding columns without a default value).

vertex-four• 12 years ago

Luckily, just as not all NoSQL databases are the same, nor are all SQL databases. If NoSQL is an option, standards are obviously less relevant, so you can rely on the behaviour of one SQL database engine of your choice. Postgres is one of a few SQL database engines which implement transactional schema changes.

Companies such as Percona also provide open-source tools which allow for online, non-locking schema changes for Postgres, and I assume commercial database engines have options as well.

davidp• 12 years ago

On the contrary, coming from the world of mature SQL systems (like SQL Server), I was horrified to learn that MySQL supports neither transactional DDL nor online schema updates. These are taken for granted in premium SQL databases, and to a lesser extent in good free ones like PostgreSQL. (Raise your hand if you've seen the polite suggestion from the Django (South?) team to use a "real" database when your migrate command fails...)

Put simply: MySQL was written by and for people whose time isn't valuable. I.e. the assumption is that it's cheaper to throw low-grade people and brainpower at workarounds (OpEx) than to pay to have the problems avoided in the first place (CapEx). There's a reason people shell out the money for big-iron databases: the math can be easy when you factor in the costs saved (both the direct costs and the opportunity costs).

overgard4• 12 years ago

Well, I think the static typing analogy pretty much cuts to the heart of it. I wouldn't be shocked if static typing advocates tend to like relational databases, and dynamic typing advocates tend to favor nosql. (I can't prove this -- but I wouldn't be surprised).

pilif5• 12 years ago

One data point: I prefer dynamic typing in programming and I'm totally in love with static schemas.

threeseed12• 12 years ago

Nope. MongoDB is very popular in enterprises which are largely dominated by Java i.e. strongly typed. MongoDB actually suits strongly typed languages since you enforce your domain in code rather than in the database.

And since Morphia (https://github.com/mongodb/morphia) is similar to JPA it is trivial to port your existing code to MongoDB. Which then leaves you with the same experience as an SQL database except with a lot of benefits (single binary, easy to query/administer/scale, geo spatial features etc).

rwallace• 12 years ago

I don't think so. I'm a fan of Python and Postgres. Rationale: there's nothing special about type errors in the code. They're just ordinary errors; you fix the code and move on. But if bad data gets into your database, it's too late to just fix the code. By the time you discover the problem, you're already set up for a very bad day, or long sequence thereof. The database has to be treated as sacred.

gordaco• 12 years ago

I agree, and I'm pretty sure there is a correlation between those two axes, although there are a lot of subtleties that can make someone prefer dynamic+SQL or static+NoSQL. Speaking for myself, I'm completely in the static+SQL camp.

I suspect that it would be somewhat harder to find static+NoSQL combinations than dynamic+SQL, since it's easier to use a dynamic language over an SQL database (types in SQL don't match 1:1 those in languages anyway, so queries are traversed by column index in both static and dynamic languages; this means that SQL's "static typing" won't affect your code) than a static language over a NoSQL database (suddenly you start to lose guarantees about what goes in every field, and checks must be performed).

hga• 12 years ago

A good relational database that implements the normal checks and constraints can serve as an additional check on your dynamically typed code trying to store gubble in the database.

jeffdavis4• 12 years ago

I agree that the philosophies seem to match up, but I believe most practical people try to look for machine guarantees from somewhere to contain the complexity.

Someone who is comfortable with strong guarantees in the database may be more willing to use a dynamically-typed language for the application, and (less commonly) vice-versa.

collyw2• 12 years ago

I had the same thought about NoSQl / SQL being like dynamic / static languages. But I prefer dynamic langauges. I think I like the strictness that a relational database gives me, so that I can be worry less about the app code level (which will be in a dynamic language).

threeseed12• 12 years ago

It's because a fixed schema is unnecessary in most cases. Most developers using a database are doing so via an ORM. In which case they already have a data model in their code. It often fixed and depending on the language strongly typed. It is also centralised, richer, commented, has far better validation and in source control.

So again why do I need a second data model that does nothing but get in my way ?

pilif5• 12 years ago

because querying and aggregating your data with arbitrary conditions becomes much easier. Using SQL you describe what result you want and let the database figure out how to retrieve it.

In many document stores, you have to write manual map/reduce jobs telling the system how to get the data you want.

m_mueller4• 12 years ago

See my other response about the advantages of schema-less design. I agree that fixed schemas can often be useful - the point is to think about this in advance for every project before deciding on SQL vs. NoSQL. I'd say a fixed schema helps during initial development, but will lead to more and more expensive data migrations later on in a system's life cycle, except in case of systems that are very well defined before anyone starts writing one line of code - and sometimes it's plain not possible if you want to have a flexible data model that's not fixed at programming time. It's simply - as always - a question of choosing the right tool for the right job.

pilif5• 12 years ago

I'd say a fixed schema helps during initial development, but will lead to more and more expensive data migrations later on in a system's life cycle

Yes. The migrations might become expensive (though that varies between databases - some take exclusive locks for a long time, others don't), but you can save a lot of time and money when querying the data.

In a reasonably normalized schema, there's next to nothing that you can't extract with the help of SQL and whatever you throw at it, databases have become very good at planning the optimal way to query for the data you've requested.

If all your data is in huge blobs, querying for arbitrary conditions and aggregating parts of those blobs is complicated and requires manual planning of every query you may come up with.

If your data is well normalized, whatever SQL you throw at it, the database will know how to answer. No specific retrieval code needed.

As such I would argue that a strict schema and a relational database will save you a lot of time as the requirements of your application change.

matwood• 12 years ago

I agree. I wonder how many NoSQL advocates should really be called NoMySQL advocates. I have always been a NoMySQL advocate, but not necessarily a NoSQL advocate.

lmm2• 12 years ago

The advantages of a fixed schema are similar to the advantages of a static type system in programming

Yes, but I already have one of those in the form of a static type system. I dislike database schema for the same reason I dislike XML schema: it tends to enforce only the most trivial constraints, and means you duplicate part of your validation (because you still want validation in code for the things you can't check in the schema). It's just another moving part that can be a potential source of failures.

Jweb_Guru2• 12 years ago

Are you sure about that? You can validate an awful lot in a PostgreSQL constraint, and I often don't bother validating in the app as a result. Validation in the app can also be extremely difficult to get right in a concurrent environment. There is a high likelihood that many such checks have race conditions that you aren't aware of, if you haven't gone over your application with a dedicated DBA.

ionforce2• 12 years ago

I don't agree with you and I'd like some clarification from anyone who knows for sure.

If the database is small, then sure, schema changes are harmless. But if a database is in production with data shuttling in and out of it and has many records, then ALTERs are non-trivial.

And I don't think an ALTER can survive in a transaction, i.e. it is not something that can be rolled back. It can be undone via another ALTER that is it's opposite, but it cannot be rolled back the way INSERT and UPDATE can.

JonnieCache3• 12 years ago

postgres can rollback everything except advancing sequences, and dropping the whole DB.

zwily• 12 years ago

DDL can be rolled back by Postgres in a transaction.

dragonwriter• 12 years ago

And I don't think an ALTER can survive in a transaction, i.e. it is not something that can be rolled back. It can be undone via another ALTER that is it's opposite, but it cannot be rolled back the way INSERT and UPDATE can.

It can in databases that use transactional DDL (and, really, doing so is part of the relational model, since part of that model is that metadata describing the structure of data is itself stored in relations and -- whatever the syntactic sugar layered over top of it -- can itself be manipulated with the same tools that regular data is manipulated with.)

SigmundA• 12 years ago

Fixed schemas aren't just about validation but performance and space usage as well.

Consider this long open high priority enhancement to MongoDB: https://jira.mongodb.org/browse/SERVER-863

Without a fixed schema, the schema must be stored with each item. Every "row" not only has the data but the metadata as well, including field names.

If you have a lot of repeatable data this can consume very significant amounts of space (regularly on the order of 4x comparing PG rows to BSON docs). This isn't just a disk space issue, this means less caches and more pages to scan when doing any operation. Compression only improves the disk space issue while adding CPU overhead.

Relational DB's like PostgreSQL having a fixed schema need only store it one time, each row has minimal overhead (about 23 bytes in PG) to store only the unique data.

Perhaps Mongo could optimize behind the scenes, detecting a repeating schema and making a fixed schema data structure for it similar to how the V8 javascript engine creates C++ classes for javascripts objects so its not a key-value lookup everywhere limiting performance. I haven't seen any NoSql db attempt that, I don't think any even attempt simple string interning as suggest in Mongos 863 issue.

PG gives me a choice I can go nearly schemaless with the JSON type or I can create a fixed schema when that makes sense, with many optimized data types to choose from. PG having array support give you a nice middle ground between the two as well.

justinsb2• 12 years ago

My personal "big picture" critique of MongoDB is that I see it evolving into a SQL database with a different syntax. It is a strongly-consistent, distributed-through-replication-and-sharding system. Additions to 'traditional' databases, like Postgres' HStore or MySQL's HandlerSocket show that many of the MongoDB differences are not fundamental.

Much more interesting to me are systems that do something fundamentally different. e.g. explore different parts of the CAP trade-off, like Cassandra. Or systems that are built around the idea that data no longer lives exclusively on a server, like CouchDB.

danpalmer2• 12 years ago

I think I disagree with you on a few points. MongoDB might be adding features of SQL databases, and heading in that direction, but they are still missing joins, and that's a crucial part of relational databases.

Also, I'd clarify a little by saying it's trying to be a strongly consistent, distributed through replication and sharding database. Right now there are some bugs in the system, some improvements that need to be made, and definitely some default configurations that need to be changed before it's going to be a consistent database. It's definitely going for consistency instead of availability, but I don't think it can call itself consistent yet.

justinsb2• 12 years ago

I don't think we disagree as much as you might think!

I'm trying not to critique MongoDB today, but rather thinking of what it will become. Today's implementation problems will hopefully be fixed.

I agree that without joins, MongoDB querying is very limited. I credit them enough to believe they will end up fixing that and supporting joins in some way. But then, they have "SQL with a different syntax".

And I definitely agree on the data-loss bugs. But again, presuming they fix those, they just end up with something very similar to a relational DB, in terms of the CAP theorem etc.

In short, even when/if they fix all the bugs and add all the missing features, I worry that we'll just end up back where we started.

overgard4• 12 years ago

Well, if that's truly the case, there's no reason you couldn't just translate SQL into JSON queries. (It's been a while since I looked at .NET mongo drivers, but LINQ in .NET basically already does this).

hartator• 12 years ago

It´s funny to see this kind of posts now and then predicting the close end of mongodb... for several years now!

MongoDB is here to stay. It´s opinatred, PostrgreSql isn't. It's faster out of the box. The client drivers are pretty good. (Dont forget that SQL database still send raw text as request and get raw text in return!). It's fitting the bill for a lot of quick and dirty web apps and deliver early performance and argualy scalable performance. Dont get me wrong, I still litteraly love postgres.

simonw2• 12 years ago

opinatred?

orthecreedence• 12 years ago

It's faster out of the box

...for reads. Once you get any real traffic it hangs on its DB write lock. Yes, I have experienced this, and the solution "just shard!" is especially obnoxious because setting up sharding is a lot harder than they make it seem. Not only is it complicated, you go from one server to three config servers, two repsets with 3 servers each (so 9 fucking servers) just so you can make more than 40 writes/s that most relational DBs wouldn't bat an eyelash at.

It's a poorly designed system, and this becomes painfully apparent once your app moves out of the basement and into the real world. Luckily for MongoDB, most of their users' apps never leave the basement and they think "wow, this is great!!"

argualy scalable performance

Not scalable. At all. Global locks do not scale. Complicated config setups are hard to scale.

Check out RethinkDB. It's an open-source document DB that fixes just about all the problems MongoDB has and it has DB-side joins. It's just as easy to get quick and dirty apps running against, and it doesn't actively flush your data down the toilet like Mongo has been known to do.

robertsosinski• 12 years ago

"Dont forget that SQL database still send raw text as request and get raw text in return!"

Depending on your database adapter, this is not the case. With Postgres, you have the Simple Query Protocol and the Extended Query Protocol (http://www.postgresql.org/docs/9.3/static/protocol-flow.html). A Simple Query involves text in and out. However with an Extended Query, properly parsed statements are always executed by the DB, and will always return properly typecasted data. You can also stream data to the client and use Cursors, if you have very large datasets (a life saver when reporting on large tables). Also, you don’t need to worry about SQL injection when using Extended Queries, as they are handled as prepared statements.

JDBC supports the Extended Query Protocol with Postgres, so if you are using Java, Scala, JRuby, or anything running on the JVM, you are good to go. But, I have not had to deal with raw text being returned from my Postgres queries in a long time ;)

gdulli2• 12 years ago

"This is not to deny that MongoDB offers some compelling advantages. Many users have found that they can get up and running on MongoDB very quickly, an area where PostgreSQL and other relational databases have traditionally struggled."

A few things about this I've never understood.

1. Someone's going to make a technology decision with priority given to a one-time cost over the strengths and weaknesses of running competing products in perpetuity? The ability to avoid having to learn something is a pro in this decision?

2. If you don't have the skill set to install or maintain MySQL or Postgres, you should not be in charge of installing or maintaining production systems, period. You will hit your ceiling the first time you have to do anything non-trivial with a system that happens to have been easier to "get started with."

kristiandupont2• 12 years ago

If you are building a system for a fortune 500 with a well defined purpose, that will run for years, your #1 might apply. But if you are building something new that you aren't sure what will become yet, like a startup, I would go with the thing that gets you up running the fastest. This allows you to validate quicker and then perhaps change if the project grows to require it.

As for #2, I think that technology is now such a big field that this sort of argument is very rarely valid. NoSQL databases might be so young that there are no db admins with that as their primary field just yet, but that will change soon.

prawks2• 12 years ago

That quote doesn't directly relate to either of your points. It's only pointing out that an advantage is that Mongo is easy to get set up and load data into quickly.

In certain use cases, that advantage is important (rapid prototyping). In other cases, it's easily outweighed by things like the potential lifetime of the system and the needs associated with that.

No one should make a decision on which tech to use by any one strength/weakness in isolation, and I don't think this article is promoting that mindset.

astalwick2• 12 years ago

The critical point in the life of a startup is very often that short period at the start, when the team is small and stretched in a hundred directions, trying to 'execute' on everything at once. It's really important to choose your battles.

Where I work (wavo.me), we went with MongoDB, and though imperfect, it was the right DB for us. It's quick to get going, and the schemaless document store makes it easy to change directions and iterate very quickly. That was (and still is) super important to us. It's really not about skillset, it's about cycles.

gdulli2• 12 years ago

My point is that MySQL/Postgres aren't difficult and this is a pretty low-hanging fruit example of a decision that doesn't need to be made and shouldn't be made with short-term thinking. Maybe it's necessary at times, but you can go too far with that excuse. Code is much, much easier to change/fix than a data store.

angrybits2• 12 years ago

In a non-trival application, using a schemaless document store doesn't relieve you of having a schema, especially in the face of a relationship change. You either have to write a migration to change all the records, which non-MySQL relational engines happen to be extremely good at, or you litter your ORM with a bunch of version specific nonsense.

m_mueller4• 12 years ago

I can't really speak about Mongo, but since the post seems to be talking about relational vs. document based DBs in general, here's my perspective coming from CouchDB:

- Schemaless DBs make sense, when handled correctly within the application framework, for what I'd call information systems with regularly changing requirements. I'm currently building a rapid development platform for these kinds of systems, where users can define an arbitrary relational data model from a Web-UI and get the application with forms and views all pre-made. The user design is changeable at any point without breaking anything and even the underlying static data structures can be changed without any need for an update or data migration process - it's all handled when opening or saving a document with a previous version.

- CouchDB's map/reduce view definitions are interesting when designing a DB system, since they IMO restrict the developer in exactly the right way: One is forced to write the right kind of index views instead of being able to just join willy nilly. Making something slow usually means writing complex views and chaining many lookups together - one has to work for it and, conversely, being lazy and reducing everything to the simplest solution usually results in a fast solution as well. The result usually scales up to a high number of documents in terms of performance.

- Being able to replicate without any precondition, including to mobile devices with TouchDB, is a big plus - and in fact a requirement in our case. Offline access is still important, especially in countries where people spend a lot of time in trains or for systems that manager types want to be accessing in flight.

SideburnsOfDoom3• 12 years ago

Schemaless DBs make sense ... for some cases

I think his real point is that on a SQL database like PostgreSQL you can add schemaless parts to your data if you think that it's right for your case, and with other DBMSs "capabilities in this area are improving rapidly".

But starting with a "Schemaless db" it's much harder to go in the other direction. "The advantages of relational databases are not so easily emulated."

m_mueller4• 12 years ago

Yes, that point is true - it's easier to go from the stricter environment to the more open one - I guess that's basically just how entropy works. Note that I don't necessarily disagree with the article, I just wanted to offer some perspective on why I think the schemaless approach can often make sense, especially in how CouchDB handles it. If at some future point, the worlds of RDBMS and NoSQL fold into one DBMS handling all the use cases perfectly, I'm all for it - it's just that right now I'm still skeptical about going all schemaless on an RDBMS, since it's still a rather new feature, and the underlying system is very complex and has grown for decades.

overgard4• 12 years ago

Having used mongo in a professional context, I'm sort of amused by how much vitriol it gets. It has it's flaws, but it's not that bad. I think it's been a bit misbranded as the thing you use to "scale", which ticks people off. To me, when I use mongo, I mostly use it because it's the most convenient option. It's very easy to develop against, and awesome for prototyping and a lot of times it's good enough that you never need to replace it after the prototype phase.

Relational databases are great, but they're almost an optimization -- they're way more useful after the problem set has been well defined and you have a much better sense of data access patterns and how you should lay out your tables and so on. But a lot of times that information isn't obvious upfront, and mongo is great in that context.

brainlock2• 12 years ago

Relational databases are great, but they're almost an optimization -- they're way more useful after the problem set has been well defined and you have a much better sense of data access patterns and how you should lay out your tables and so on. But a lot of times that information isn't obvious upfront, and mongo is great in that context.

I think it's exactly the other way around. I prefer to lay out tables in a way that reflects the meaning and the relationship in the data. Only later, if there is a bottleneck somewhere, I might add a cache (i.e. de-normalize) to better fit the specific data access patterns that were causing trouble.

threeseed12• 12 years ago

You missed the point.

Since most companies now are doing Agile development there isn't the big upfront design process where the data model is clearly understood at the beginning. Instead you have this situation where the schema is continually evolving week by week and hence this is why schema less systems can be appealing. It isn't about performance.

prawks2• 12 years ago

I think if your understanding of the domain is complete enough that you can map out a relatively wholesome definition of the relationships of data then you're probably right.

I think the advantage the parent is talking about is when you're exploring a domain that you do not have as deep of an understanding in. Schema-less data storage can be helpful in that exploration, as it allows you to dive in and get messy without much upfront consideration. Then afterwards you can step back with what you've learned/seen/experienced and build out your conceptual model of the problem domain.

mlieberman85• 12 years ago

I agree it has its uses but I feel like MongoDB (the company) sometimes puts forth use cases for MongoDB (the database) that are untenable. For example the pre-aggregated reports use case that they tout that fails at any reasonable scale (e.g. hundreds of upserts a second against tens of thousands of documents)

overgard4• 12 years ago

Yeah, I can agree with that. A lot of their claims of scalability and benchmarks aren't lies, but they come with a lot of asterisks. I know in my last job we used both mongo and (mainly) microsoft sql server, and frankly, I enjoyed developing against mongodb way better, but if I needed something to be consistently fast, there was no question I'd put it in sql server. (Mongo's query speed can be really variable, and the tools to diagnose query planning kind of suck compared to more established databases)

threeseed12• 12 years ago

A lot of the vitriol seems to come from the PostgreSQL camp. The enemy used to be MySQL but since they lost that fight they seem to enjoy spreading FUD about MongoDB. If they just focused on improving their own product i.e. easy of use, sharding, clustering etc. there wouldn't be a need for MongoDB or even HBase and Cassandra.

worklogin• 12 years ago

Postgres is winning the war with MySQL, if you're going to classify it that way. And from everything I've read and heard on HN, Postgres is constantly improving on clustering/HA fronts.

kristiandupont2• 12 years ago

To me this whole discussion seems equivalent to the static/dynamic typing war in programming languages. And so far, none of the camps have "won" as has been predicted over and over again. There are tradeoffs to both and people will prefer one over the other for what they will believe are rational reasons but probably really just a series of coincidences.

opendais2• 12 years ago

Most of the vitrol stems from the fact that MongoDB's default settings were unsafe for a very long time and people were burned by it. Imo anyway.

threeseed12• 12 years ago

Which is nonsense since every driver since the beginning set the default to be safe.

Most of the vitriol comes from people who never even used it.

jeffdavis4• 12 years ago

Informed, perhaps even constructive, criticism is not "vitriol".

jeffdavis4• 12 years ago

"Relational databases are great ... after the problem set has been well defined ... But a lot of times that information isn't obvious upfront, and mongo is great in that context."

A lot of people get stuck on the schema because they are trying to make it perfect upfront and hit analysis paralysis. I think that comes from other database systems that make it very hard to change the schema.

In postgres, just define a very simple schema and get started with your prototype. It may just have a couple ordinary columns, and then a JSON or JSONB column as a catchall.

Then, you can add/remove/rename columns and tables (all O(1) operations) or do more sophisticated transformations/migrations. All of this is transactional, meaning that your app can still be running without seeing an inconsistent (half-migrated) state. I believe this level of flexibility far exceeds what document databases offer.

mathattack• 12 years ago

I think many folks confuse normalization as a strategy with the underlying database technologies. Oracle and other RDMBS technologies can create normalized databases too. In the end it's a design judgment. There is a lot of room between fully normalized and one-big-table. Even firms that logically map things out fully normalized frequently decide that for some things that doesn't make sense.

Taking a step back, there are still reasons to abandon Oracle. It may not scale up, or be good for certain time series calculations, but that's another story entirely.

loftsy• 12 years ago

On document store indexes the article says:

If all order data is lumped together, the user will be forced to retrieve the entirety of each order that contains a relevant order line - or perhaps even to scan the entire database and examine every order to see whether it contains a relevant order line.

Both of these are untrue. Author needs to read up on secondary indexes.

craigching• 12 years ago

As I always say when these sorts of articles come out (and I admit this is my use case for MongoDB and that doesn't necessarily match everyone else's use case), where is the easy to setup HA and sharding for PostgreSQL? I know it's coming, but right now it's not there.

For someone who redistributes a product that relies on end-users setting up HA, this for me is MongoDB's killer feature, easy to configure replication and sharding. I love PostgreSQL, but this is the one big thing that keeps me from using it right now.

lucisferre• 12 years ago

The author is conflating (or just ignoring) the very significant difference between application databases and reporting databases. Not surprising since most of us do this as well when we are building applications. However no comparison of the relative value of database schema styles can responsibly ignore this difference.

Yuioup• 12 years ago

In short, I don't expect MongoDB, or any similar product, to spell the end of the relational database.

The author makes it sound like that was a possibility. SQL and NoSQL are two different tools in the toolbox and should be considered as such.

danford• 12 years ago

Often when I read articles like this I take them with a grain of salt. A lot of hate for MEAN technologies seems to stem from people who don't know how to properly use them. You're not supposed to use MongoDB like *SQL and if you try you're gonna have a bad time.

Let's say I have a big pick-up truck I use to haul xWidgets. My friend gets a little motorized scooter to drive around town hauling his yWidgets. Is it proper for me to believe that my friends scooter sucks because it doesn't haul xWidgets like my truck? I don't know much about scooters, but by friend says it doesn't have a steering wheel and it only has two wheels. How the heck can it steer without a wheel? He says it has "handle bars" for steering. Seems kind of dumb but I guess it works for him. He touts the fact that he gets 80mpg on gas but it doesn't matter how efficient it is if he can't haul xWidgets. Scooters suck because they don't work the way my truck does.

Jweb_Guru2• 12 years ago

Okay. What does this have to do with MongoDB, though? People in this thread have presented plenty of usecases for it, but every single one of them (pretty much--someone correct me if I'm wrong) is also satisfied by jsonb in Postgres 9.4, which in your analogy would make it a truck that gets 85mpg.

jchrisa• 12 years ago

Maybe the clock is ticking because thier large production deployments are migrating to other tech? At least we are seeing plenty of folks who realize that a query API on top of mmap isn't really a database. :)

One high profile migration: http://www.couchbase.com/viber

threeseed12• 12 years ago

MongoDB will be shipped as part of every RHEL installation. So pretty sure it is actually going to be getting even more popular.

ThePhysicist2• 12 years ago

I agree that document-oriented databases will probably not replace relational databases in the near future. In my opinion though, the schemaless design of MongoDB paired with its ease of use and its native support of JSON data makes it a perfect choice for prototyping and (in some cases) a viable option for use in production.

What you also have to consider when comparing document-oriented to relational databases is that the former is still a very young technology: MongoDB has been founded in 2007, whereas Postgres has been around since 1986! So given what the MongoDB team has achieved in such a short time span, I expect to see some huge improvements in this technology over the next decade, especially given the large amount of funding that 10gen received.

In addition, the root cause for most complaints ("it doesn't scale!", "it loses my data!", "it's not consistent!") is that people try to apply design patterns from relational databases to MongoDB, which often is just a horrible idea. Document databases and relational databases are very different beasts and need to be handled very differently: Most design patterns for relational databases (data normalization, using joins to group data, using flat data structures, scaling vertically instead of horizontally) are actually anti-patterns in the non-relational world. If you take this into account I think MongoDB can be an awesome tool.

theseoafs2• 12 years ago

To be fair this particular complaint:

"it loses my data!"

is entirely legitimate. A database product should not be losing data unless you specifically allow it to. Period.

laichzeit0• 12 years ago

There's cases where occasional data loss is an acceptable trade off for performance. In a particular app I'm working on it's ok for me to have 99.9% of the data and all my updates and inserts have write concern set to 0, journaling off, etc.

camus22• 12 years ago

In addition, the root cause for most complaints ("it doesn't scale!", "it loses my data!", "it's not consistent!") is that people try to apply design patterns from relational databases to MongoDB, which often is just a horrible idea.

Is expecting a database to be acid a horrible idea? if you care about your datas it is not.

ThePhysicist2• 12 years ago

No that's probably not a horrible idea, I'm just saying that MongoDB is by design not an ACID database, so I think it's silly to complain about this fact when using it.

AdrianRossouw2• 12 years ago

I've been asking a lot of people when MongoDB is actually the right tool for the job.

http://daemon.co.za/2014/04/when-is-mongodb-the-right-tool

I'm starting to form this idea of what constitutes an ideal use case for mongo in my head, and i'm trying to prove the model.

If I were to imagine some kind of "realtime" multiplayer game, like quake or something.

1. You have to have the state be shared between all the parties in a reasonable time.

2. The clients only need the data that is directly relating to the round they are in, so you have the concept of cold and hot data.

3. The data is all kind of ephemeral too, so that you don't specifically care about who was on what bouncy pad when, but you do want to know what the kill score/ratio is afterwards.

4. You have a couple of entities that have some kind of lightweight relationship to each other, which makes it just more complex than a key-value store like redis is really suitable for.

5. These entities are sort of a shared state, and thus get updated more often than new unrelated documents get added, and couchdb's ref-counting and append-only nature makes it really unsuited for constant updates of an existing record.

any feedback would be appreciated.

drhayes9• 12 years ago

This seems totally reasonable to me. I was playing around with using Firebase (https://www.firebase.com/) as the datastore for a turn-based multiplayer game on the web and its data policies seemed very well suited.

Essentially, each game was its own document and relevant stats were updated/duplicated on the player documents as well.

There don't seem to be useful relationships here that I'd want to query. Maybe successive game states? But not across different games, right?

AdrianRossouw2• 12 years ago

Well, I was imagining a situation where you had multiple different pieces in a game, relating to each player and distinct to that game. Maybe like an RTS game or something?

So you might want to query how many pieces a certain player has available, or do a check on who this piece belongs to.

Keep in mind I'm still not saying an actual game, but any problem that had similar data storage and access patterns as a game along these lines would have.

I don't know enough about firebase to confirm whether it is suited to this, but if I had any kind of backend at all, and I had requirements for really quick reponses, i wouldn't be comfortable adding the extra hop to a hosted service onto each call.

yawz• 12 years ago

Dramatic titles get the readers attention, therefore I get the choice of words. However, our industry is so big that there will never be a single solution. MongoDB may never become #1 but as it's described in many comments, it's a pretty good choice in various situations. So, as they say in Ireland "Stall da beans der bi!" I don't hear a ticking clock.

joeclark772• 12 years ago

From reading Redmond & Wilson's book "Seven Databases in Seven Weeks", my impression is that the CAP Theorem is a big idea (essentially, you get two choose any two: consistency, availability, and partition-tolerance). Relational databases have to be "consistent", i.e. transactions are atomic and reads always get the latest value. NoSQL databases make it possible to choose Availability instead of Consistency, so you can get faster response times even if it means you sometimes get out-of-date data.

I would use Mongo or Riak or something like those for "write once, then read-only" applications, for example a twitter or facebook type application, or simple blog. In those cases when the user hits "refresh" they're never going to know or care that they don't yet see a comment someone posted a half second ago. They're just going to be happy that the page refreshed fast.

EGreg• 12 years ago

Actually, graph databases are better suitsd for many of today's social applications.

remon• 12 years ago

Given the title I expected a completely different article. The reason being that the opinion that "MongoDB is running out of time" can be supported by a few very strong arguments. There are a few very serious problems with the current technology that give NoSQL alternatives a chance to catch up and grab momentum. For example, MongoDB still has a questionable mmap based storage engine, there are cluster state consistency issues, the cluster topology management is outdated and so on. This article should have been titled "Why I personally like relational databases better than MongoDB". That discussion is incredibly subjective. Source : Very early adopter of MongoDB and use it professionally.

joeclark772• 12 years ago

It was a play on the title of the article that the author was responding to.

ulisesrmzroche2• 12 years ago

Man, how long has it been since MongoDB has supposedly been dying? I swear it's more than 3 or 4 years for sure.

Why The Clock is Ticking for MongoDB

Related Stories