(Some) ORM haters do get it

http://www.revision-zero.org/orm-haters-do-get-it

by blambeau • 14 years ago

116 105 14 years ago

ianterrell• 14 years ago

The author's claim that ORMs are bad computer science is probably accurate. Fortunately, they're really good engineering.

Roboprog• 14 years ago

We could be more subtle, and go with "it depends". For getting and putting data to be used on an edit form, using an object/class which actually has (non relational constraint) "business logic" in it, ORM good. For batch mass update, ORM bad. (one wonders if every table needs a custom class, though)

richardlblair• 14 years ago

Batch updates aren't where it ends. As a Django developer with a strong SQL background I find my hands are tied far more often than I would like. Sometimes this is caused by bugs like the current group by bug[1], other times it's caused by the design of the ORM.

I do agree that the ORM is handy for things like "Get me all the things in this table", and "update this single record using a form", but this author is dead on. There is logical reason behind the hate developers have for ORMs.

[1] https://code.djangoproject.com/ticket/17144

mistermann9• 14 years ago

Can anyone point out the coup de grâce the author seems to think he has arrived at?

He points out some (well known) ways that ORM's can be used inneficiently, and acknowledges the techniques that have been developed to work around these, but then seems to conclude that he has proven once and for all that ORM's are bad. I totally missed the connection on that part. Is it that SQL is better in dealing with sets than an ORM (a fact no one denies), therefore you should not use an ORM?

sgift• 14 years ago

The connection (according to the author as far as I've understood him) is: "All the techniques that have been developed to work around the inefficiencies of ORMs are just reinventing the wheel. The solutions have been there for 40 years and your workarounds are only needed because you think about individuals (object-orientied) instead of sets (relational)."

stcredzero2• 14 years ago

your workarounds are only needed because you think about individuals (object-orientied) instead of sets (relational).

What if it's better structurally for the program as a whole to think about individuals?

mistermann9• 14 years ago

> The solutions have been there for 40 years and your workarounds are only needed because you think about individuals (object-orientied) instead of sets (relational)

(This is to OP rather than you)

Well if that's the argument he's making, one example I can think of, in the case of an extremely complex update, while it always can be done in pure SQL, it is much easier to logically code using an ORM, perhaps even using individuals rather than sets (the horror). And while this implementation might execute slower (.1 second vs .01 second), it is vastly simpler to read and refactor without screwing something up (ie: economically cheaper), and as for the performance argument, it only needs to be fast enough.

nahname3• 14 years ago

Which is creating a false dichotomy. Assuming your are using some OO language then you will have to, at some point, turn the relational data into objects. At that point you are always creating an ORM.

There is a really interesting post about how writing that code is literally stealing from your clients. I tend to agree. http://ayende.com/blog/3712/stealing-from-your-client

chrisrhoden• 14 years ago

I think his argument is basically the same as the argument that people who put down things like Redis use: You're just slowly recreating SQL. I don't think it's accurate, but that seems to be the argument.

dcminter• 14 years ago

There is a need to manipulate relational data from object oriented code. ORMs are tools that facilitate that. The Object/Relational impedance problem doesn't go away if you hand-carve the code, it just makes you work hard on all the points of contact instead of just the problematic ones.

The real "problem" with ORM is when people use such tools as a way of avoiding having to understand databases (and specifically SQL). Fortunately that's becoming less common at least within the Enterprise Java world where I live and breath.

mb_72• 14 years ago

I think you've hit on something here. I came to start using ORMs after 10+ years of writing / optimising databases and SQL. When using an ORM (and most of my work is done with a probably not-well-known one, XPO from DevExpress) I'm aware of what is (or should!) be happening under the hood; my prior experience with 'bare metal' is extremely useful, nay essential to creating a performant system. On occasion, it's necessary to do a direct SQL query, but not often.

Sure, the apps I'm writing are for small-medium business, but XPO 'just works'. Context is important; if I was working on something with more users / tighter speed requirements, an ORM may or may not be the best choice. Still, this falls into the 'right tool for the job' category that good developers are already aware of.

FuzzyDunlop• 14 years ago

I'm well open to correction on this, since I've not much of a clue, but with all this ORM back-and-forth, and relational databases, why do we not see more usage of graph databases?[1] From the wiki, it says they map more directly to OO applications. Is there a reason relational databases are still used by default?

[1] http://en.wikipedia.org/wiki/Graph_database

nknight• 14 years ago

Inertia, accidents of history. Relational databases had obviously-useful properties at critical points in the history of mass commercial adoption of computing. As a practical, real-world matter, "object-oriented programming" barely existed during the rise of relational in the 70s, making the practical problems caused by the impedance mismatch either non-obvious or seemingly unimportant.

As a result, huge investments were made in advancing relational databases, and other forms have languished. Today, relational maintains the substantial advantages of maturity and installed base. Performance, reliability, general polish, ease of access to support/tutorials/other literature, and general status as "the way things are done".

biafra• 14 years ago

Sometimes the reason is that operations is used to it.

ExpiredLink• 14 years ago

This article is very much to the point. ORMs use the wrong abstractions. They try to 'map' 4GL to 3GL. The results are necessarily unsatisfactory.

soc883• 14 years ago

Absolutely right!

debacle5• 14 years ago

I don't understand where this ORM 'divide' is coming from.

ORMs are powerful, because they let you say less and do more. For 90% of the queries out there, an ORM is fine.

SQL is powerful, because you can control and fine-tune your statements. For the remaining 10%, use SQL.

Are ORMs bad? No. Can you them for everything? No.

The same thing can be said for almost every technology in existence.

lloeki• 14 years ago

> ORMs are powerful, because they let you say less and do more

To me this is a bonus. The real deal is that ORMs allow you to write queries as "first-class" components of the language, hence benefiting from language features such as type checks, duck-typing, factoring, static analysis even, and more.

Compare this to stitching strings (however parametrized they are) and manually coercing your object values to strings.

This is where, IMHO, ORMs like ActiveRecord and Arel shine, as they give you access to each building block (form connection.execute to .quoted_table_name to .to_sql) so that you can place yourself at whatever level of abstraction between the two worlds you may need.

nahname3• 14 years ago

Don't forget the whole other side of it.

http://en.wikipedia.org/wiki/Object-relational_impedance_mis...

Retric• 14 years ago

IF ORM's stuck with the 80-90% case I don't think people would have problems with them. Unfortunately, they try to handle an ever wider range of edge cases which tends to create complexity and cruft. As soon as I need to double check their output and I see horrible gobblygook for a moderately complex query I am generally better off writing it myself than trying to understand how I can get the ORM to make a cleaner query.

Still, ORM's seem to be getting better Linq to Entity Framework seems to work much better than a lot of the old ORM's I have used.

debacle5• 14 years ago

ORMs don't try and handle anything. People apply them poorly which simply creates bad code.

The same can be said of the goto statement, or any number of other things - used improperly, they screw up everything, but in some instances they are necessary and good.

asolove• 14 years ago

I agree that the 'divide' is overstated. But the author does mention one frequent problem: when you begin a project in an ORM, you may begin to depend on validation or update callbacks that get triggered by the ORM. Later, if you discover queries that need to be written manually, you will first likely forget these rules, then have a hard time translating them into SQL, and then have an enormous, permanent headache keeping the two in sync.

gbog3• 14 years ago

"ORM is fine" 90% or even 100% of the time don't make them right. Thee problem is that OOP is not well defined, it belongs to "soft" science, and relational databases are "hard" science. I have read the article about ORM and Vietnam, in we just have all of the symptoms of impedance mismatch.

I have to add that this issue is much deeper than a simple efficiency issue. A perfect db with 100% availability would still not be well matched to an object system.

I listened to a podcast from 1990, the guy said "the problem with objects is that we don't know what they are". I just finished SICP, and it seems it is the same conclusion: objects with their state, their messages, their multiple parents, their instanciation, are not a proper model for many or most cases in software development.

Example of proper models include types, text files, streams (pipes), modules, maybe services.

ajross• 14 years ago

Related to your point would be an argument that software engineering is more productive when treated as a "hard" science (which I take to mean something like "rigorous justification of all steps" or "clear derivation from principle").

I don't think that's true at all, as evidenced by pretty much every successful software product ever (including the early relational databases, I should add) but would be curious to see the counterargument.

romaniv• 14 years ago

Set theory is hard science. On the other hand, a grapevine of stored procedures with no code reuse beyond copy-paste aren't any kind of science. I'm not even sure they are any kind of engineering.

fusiongyro7• 14 years ago

The divide is fundamentally about choosing what's in charge of your system, the system being composed of your databases, your applications, and your supporting infrastructure (your scripts, your migrations, etc.) To relational database folk such as myself, the central authority is the database, and our principal interests are what ACID exists to provide: concurrent, isolated, atomic transactions that cannot be lost, on top of a well-defined schema with strong data validity guarantees. To us, what's most important is the data, so everything else must serve that end: the data must always be valid and meaningful and flexible to query.

The side that argues for ORM has chosen the application, the codebase, to be in charge. The central authority is the code because all the data must ultimately enter or exit through the code, and the code has more flexible abstractions and better reuse characteristics.

The reason for the disagreement comes down to disagreement about what a database is about. To the OO programmer, strong validation is part of the behavior of the objects in a system: the objects are data and behavior, so they should know what makes them valid. So the OO perspective is that the objects are reality and the database is just the persistence mechanism. It doesn't matter much to the programmer how the data is stored, it's that the data is stored, and it just happens that nowadays we use relational databases. This is the perspective that sees SQL is this annoying middle layer between the storage and the objects.

To the relational database person, the database is what is real, and the objects are mostly irrelevant. We want the database to enforce validity because there will always wind up being tools outside the OO library that need to access the database and we don't want those tools to screw up the data. To us, screwing up the data is far worse than making development a little less convenient. We see SQL not as primarily a transport between the reality of the code and some kind of storage mechanism, but rather as a general purpose data restructuring tool. Most any page on most websites can be generated with just a small handful of queries if you know how to write them to properly filter, summarize and restructure the data. We see SQL as a tremendously powerful tool for everyday tasks, not as a burdensome way of inserting and retrieving records, and not as some kind of vehicle for performance optimization.

At the end of the day, we need both perspectives. If the code is tedious and unpleasant to write, it won't be written correctly. The code must be written--the database is not the appropriate thing to be running a web server and servicing clients directly. OOP is still the dominant programming methodology, and for good reasons, but encapsulation stands at odds with proper database design. But people who ignore data validity are eventually bitten by consistency problems. OODBs have failed to take off for a variety of reasons, but one that can't be easily discounted is that they are almost always tied to one or two languages, which makes it very hard to do the kind of scripting and reporting that invariably crop up with long-lived data. What starts out as application-specific data almost invariably becomes central to the organization with many clients written in many different languages and frameworks.

We're sort of destined to hate ORM, because the people who love databases aren't going to love ORM no matter what, and people who hate databases will resent how much effort they require to use properly.

mattmanser• 14 years ago

But the truth is the days of DB being king are almost gone.

These days you just don't hear about DBAs at all any more. You used to see constant jokes about DBAs being a pain in the ass and stopping programmers doing X or Y. ORMs going to win because there aren't enough of you left. Stored procedures, triggers, etc. are going to be viewed as ancient technology back from the days of yore when people didn't understand how to code properly.

mistermann9• 14 years ago

To me it sounds like you're talking about the argument of who has the responsibility of applying business rules, the application layer or the database layer - which is another entirely valid argument in itself, but distinctly different than the argument of whether to use an ORM in your application layer or not.

debacle5• 14 years ago

Your response is long, but altogether too subjective, philosophical, and tautological.

What matters is consistency, usability, and agility. Throwing ORMs out the window will give you as much consistency as you can squeeze out of an SQL server, but will greatly reduce your agility. Using an ORM for everything will greatly increase your agility but will reduce your usability.

As in everything, there is a balance. People who fall on either side of that balance need to back away from the pulpit and rethink their stance.

jshen2• 14 years ago

"To us, what's most important is the data, so everything else must serve that end"

Surely this is a flawed world view!

DenisM2• 14 years ago

This is a fantastic comment, you need to post this as a blog post, and then submit to HN. This way I and many others will never have to rehash this again, just point back to it.

adisesha• 14 years ago

Is it Man or Woman important for successful marriage? Your comment sounds similar to this :) I do not hate either. ORM or not, developers dealing with db data needs to understand how it works. If they do not, they may still come up with working application but when performance issues come up, they become deer in headlights.The same argument applies to ORM or any other technology. Long time Hibernate user here.For me, Martin Fowler's article hits the nail.

j-kidd• 14 years ago

> We're sort of destined to hate ORM, because the people who love databases aren't going to love ORM no matter what, and people who hate databases will resent how much effort they require to use properly.

Speak for yourself. I love databases (note the plural form) and love ORM. ORM is a godsend for developing application that has to work against different databases (postgresql, mssql, db2, etc).

dirtyaura2• 14 years ago

I'm running a sales reporting and payroll software written in Django, and it's more like 10% of views benefit from ORM and for 90% views ORM is hindrance and a serious performance hit.

I see ORM vs relational style as a development mindset issue. We have moved using SQLAlchemy Core (not the ORM part) and the difference in development style is stunning: now it's quite easy and more importantly FUN to write performant queries, whereas when we were using Django ORM, it was very easy to write non-performant code and tedious to make it fast.

debacle5• 14 years ago

Yes, I could see that - writing reports is one example of an instance where ORMs are almost entirely useless.

euroclydon• 14 years ago

Well now I'm reminded of my initial though when a colleague first introduced me to an ORM: "It feels wrong to use this tool to just map every object to a table." Of course I went on to write many, at best, moderately complicated web apps very fast using NHibernate and didn't miss writing vender-specific SQL or column to property mapping boilerplate.

But this article is a breath of fresh air. I may just try Dapper for the next project

http://code.google.com/p/dapper-dot-net/

koide2• 14 years ago

I agree, microORMs are the right solution if you are trapped in the mainstream. I'm a very happy user of Dapper.

bitdiffusion• 14 years ago

I think the other point the author makes is that it's not possible to write efficient code that is entirely abstracted from the underlying data (see his loop examples).

i.e. if you have to write your code in a specific way to make the ORM behave correctly (constantly thinking about what kind of sql your code is generating), then the abstraction becomes a lot less useful.

scotty79• 14 years ago

For easiest things SQL is cumbersome.

Getting single row by primary key which is 90% of access is overly verbose in SQL so ORM wins.

For slightly more complicated cases SQL is much faster and easy to write so people who have to increment field in all rows that satisfy simple condition go: "ORM sucks".

But for more complicated cases like trimming data tree in some places SQL quickly becomes too much of a puzzle for most programmers to deal with so they prefer ORM again because it's doable there and most of the times works. Dedicated SQL users who are not good at puzzles in such cases write full fledged program (if their SQL dialect allows for that) and instead of bringing data to their iterative or recursive programs they bring their programs to the data which creates hard to debug, unreadable often unversionable monstrosities.

There should be some merge between databases and programming languages that could combine beauty of syntax of modern programming languages and efficiency of massive data handling of modern databases.

Why is it ok to have standard hashmap implementation in a language but not file backed hashmap or btree index?

mckoss• 14 years ago

I think this is the motivation behind .Net Linq - bringing the relational query semantics into the language.

http://msdn.microsoft.com/en-us/library/bb308959.aspx

bkirkbri• 14 years ago

I'm pretty sure that this is the motivation behind http://datomic.com

soc883• 14 years ago

The idea of translating the declarative way of doing things to an imperative approach (that's basically what ORMs are doing) is imho a huge failure. It just never worked decently.

These days, we have languages which integrate rather nicely into the declarative mindset, so no need anymore for such bizarre "paradigm translators".

koide2• 14 years ago

What are those languages and how do they integrate nicely into the declarative mindset?

superasn• 14 years ago

I think the problem with ORM is that at the end it is only a wrapper over SQL, say like Winzip is for Zip. So while it may look pretty and easy to use, it always has to obey the rules of the host program, and so workarounds such as these have to be invented.

On the other hand if somehow ORM was part of the core compiler AND database, then somehow it could be possible that even when you write a for loop on the top and have an if conditions inside the block, or perform a join, the compiler understands it and pre-compiles your code without needing such workarounds (as there aren't two separate layers to join). So you would treat objects and objects and never have to worry about how the wrapper is being generated or what kind of indexes or queries it will run finally. I'm not an expert at compilers though but for strictly typed languages it could be possible.

pbz2• 14 years ago

That would never work though, at least not in the perfect, non leaky way the author and you would like. When you have a loop, even if the database could somehow understand that loop, you can put anything inside it. You could make a call to an outside service, write to disk, etc. With a single SQL query the database can take it and optimize it since it has a full understanding of its data domain.

kstrauser• 14 years ago

He's wrong.

SQL is great for what it does, but I use ORMs for reasons other than writing queries in a different way. I inherited an utter mess of a schema that wasn't even remotely close to 1NF. Imagine fields containing comma-joined sets of values, and with column names not even remotely related to what they actually held. For legacy and business purposes, updating the schema was a non-starter.

So I used SQLAlchemy to remap the schema into something usable. I wrote getters that split out those comma-joined fields and returned the desired value against tables that required indexes like:

    UPPER(SUBSTR(name_delpt,1,STRPOS(name_delpt,',')))

I wrote (and therefore more importantly _documented_) the bizarre and complex way some of the tables joined together. I wrote something that was unit-testable and that could be used as a foundation for other work so that I wouldn't have to memorize the insane corner cases and reproduce them from scratch each time I needed to access the data in some little-used table.

I _didn't_ write a more convenient way to say `SELECT * FROM blog`. In general, that doesn't interest me and I wouldn't have bothered with it. ORMs are great - if used well! - for encapsulating all of the little bits of business cruft in one central, easy-to-manage place. They're handy for roughly the same reasons that subroutines are handy.

thespons• 14 years ago

This is an interesting use and I like the idea, but I don't think it makes him wrong. You don't actually need an ORM to do this. The same logic could be applied to data sets retrieved through regular queries. Or you could create views/stored procedures with the same logic.

SoftwareMaven• 14 years ago

Yet another software religious war, which seems to boil down to "if you don't think my technology is correct 100% of the time, you are insulting my honor".

Do other fields get into constant pissing matches like this? Languages, libraries, process, licenses, editors, Operating systems, you name it, software engineers are fighting about how much better theirs is and how you are an idiot for not seeing the true light their vast intelligence is trying to bequeath unto you.

What is it about our brains that makes the subtlety of "use the right tool; every problem isn't a nail" so difficult? Or is it just hard wired into our need to be identified with a community?

pbz2• 14 years ago

It's probably directly proportional to one's OCD level or amount of pain accumulated through the years. There's also the need to show and feel that you're superior, and sometimes the need to point out that somebody's superiority is irrelevant.

parfe• 14 years ago

Any system I've worked in that didn't use an ORM still implements an ORM.

Save methods which runs insert or updates. GetAll methods get_by_username, get_by_id, get_by_email, get_recent, get_by_foreign_keyed_object, get_by_other_foreign_keyed_object.

And all those hand coded methods directly tied to the dialect of the db the original developer used.

Using Hibernate I had to write raw SQL for some reporting. I've never written raw SQL in Django (the project I've used django on self select for simplicity).

And using SqlAlchemy/Twisted as a backend for a desktop application I haven't found a need yet, for performance or correctness. I have one query I'm eyeing for an SQL rewrite, but it'll probably be a week of work to make sure it works correctly and I'd rather release this phase than save 30 seconds on a weekly query.

I've reached a point where ORM complaints don't really make all that much sense. The issue seems to be "THe ORM breaks down doing X and Y and Z so I had to hand write SQL!"

But you'd be writing X Y and Z anyway if you weren't using an ORM, so what's the issue?

mistermann9• 14 years ago

Exactly. Or more explicitly, the anti-ORM argument, to me, consists of: ORM works fine for A through W, but SQL is better for X, Y, Z. So rather than just the common sense solution of writing just X,Y,Z in raw SQL, everything has to be written in raw SQL.

wvenable2• 14 years ago

The argument works both ways; in fact, I think it's worse the other direction. Most projects use the ORM and just the ORM and it's against policy to write anything in raw SQL even if it's better for X, Y, Z.

ratsbane• 14 years ago

Any system I've worked with that didn't use SQL directly ended up implementing poorly a system for describing sets of data. SQL isn't a perfect implementation of relational algebra but it's better than a bunch of nested loops.

chris_wot5• 14 years ago

I agree with a lot of what you he says, however what about the case of getting 10 rows with 20 columns - each column needing a join. The optimizers often go nuts over this sort of thing, as that's 20! combinations the optimizer must iterate over to get a good join order.

Apparently, Postgres has a genetic optimizer that handles this... curious to see if this is an issue or not.

Incidentally, I'd like to say that I loved the author's Relational Basics II at http://www.revision-zero.org/relational-basics-2 I've come to the conclusion that SQL is particularly limited in its application and implementation. I'd love to see a better declarative language for databases!

nsxwolf2• 14 years ago

This article is all about RBAR vs. set based operations. It has little to do with ORMs per se. That naive ORM users may tend toward RBAR is beside the point.

Go RBAR when it doesn't matter - when it's convenient and you know how it is going to scale ahead of time. A user updating his profile. Creating an order.

Go set based the rest of the time - when you're processing a large batch of data for thousands of user accounts, offload that work into a stored procedure and call it.

ORM does exactly what it is meant to do, and if you're working with data in an OO model, you're going to be doing ORM, whether you know it or not - you will either pull a decent ORM tool off the shelf, or you WILL be writing your own very bad one.

recursive• 14 years ago

What is RBAR?

nsxwolf2• 14 years ago

Row-By-Agonizing-Row, or iterating over a list of rows and operating on each individually with a new query.

Your average Java programmer is used to iterating over collections in a while loop, where 30,000 in-memory objects can be quickly modified. It's tempting for said programmer to do the same to ORM-backed objects and issue 30,000 sql update statements across the wire.

jakejake• 14 years ago

As an author of an ORM I can agree with the OP that using them incorrectly can lead to horrible punishment on the database. He seems to be most concerned with the n+1 issue which happens when you loop through objects and another query is performed on each iteration.

If you're going to use an ORM then you absolutely have to learn the mechanics to avoid n+1 queries. Every ORM is a little different, some are easier to tune than others.

I personally favor a basic mapping that the ORM does automatically combined with an advanced mapping that allows you to basically write queries for special purposes and map them to transient objects that don't necessarily exist in your schema. You have to do this for things like aggregate queries or calls that require several joins but only need a couple of columns from each table.

The OP raises some good points but I do think that ORMs can be used properly to great effect.

Ixiaus• 14 years ago

I hated every single ORM "system" I have ever come in contact with from many different languages with the exception of one: SQLAlchemy. SQLAlchemy isn't just easy and intuitive, it is really smart and well built.

For a long time I was on the side of the ORM haters, until I tried SQLAlchemy - it truly is an awesome ORM package and I have yet to find anything like it in PHP/Ruby/etc...

iainduncan• 14 years ago

As is so often the case, IMHO the poster is ignoring the business side of the equation in favour of "correct". A good ORM is fantastic for speeding up development. If you design your db around being well usable with the ORM, the potential development gains (at least in dynamic languages like Python) are HUGE. I left Django in part because of their ORM. But using SQLAlchemy, we develop far, far, faster than if we were using SQL directly. And it's flexible enough to allow me to drop into the SQLAlchemy Query language when I need to hand tune a query, or to SQL itself if I really need to get close to the metal.

Will there be downsides one day? Probably. Will they come even close to the business value of the amount of the coding time we've saved during the critical bootstrap phase? No way in hell. As they say, those are problems I'd love to have.

skybrian• 14 years ago

This is all very nice, except that users often do want to work on individual records (at least when updating them). Codd's simplifying assumption is for mathematicians, not users.

ScottBurson• 14 years ago

There's a problem that's inherent to any client/server architecture: what work should get done on the client, and what should get done on the server? The author presents this as an ORM issue, but it has nothing specifically to do with ORMs; you'd have the same problem using an OODB server.

(Some) ORM haters do get it

Related Stories