The ORM is bad because it is a poor abstraction of SQL. This manifests itself in several ways, the largest being little control over the SQL queries generated.
Here's a gist I made the other day demonstrating this. https://gist.github.com/jawnb/cd7a899cac5300c01709
The poor abstraction really causes problems when you need to do anything beyond the simplest of use cases. To achieve even modest performance gains, you're better off hand writing your SQL. Additionally, doing even the most trivial of trivial aggregate queries will also mean that you're hand writing SQL.
SQLAlchemy is light years beyond the django ORM in this regard.
Don't take just my word for it though, here's a talk Alex Gaynor gave saying the same things. https://www.youtube.com/watch?v=GxL9MnWlCwo