Dropping Django

http://blog.brandonbloom.name/2009/08/dropping-django.html

by snprbob86 • 17 years ago

108 59 17 years ago

blog.brandonbloom.name

far33d• 17 years ago

This seems like exactly the right way to use a framework -

* start by using it as it is intended. Save lots of time by being up and running quickly.

* find a flaw.

* realize that the workaround is more work than rewriting a piece of the framework.

* stop using that part of the framework, but continue to use the rest of the code that you still haven't had to write.

jwecker• 17 years ago

Agreed. I eventually moved from django to pylons going exactly through this process- except that instead of rewriting pieces I was able to just plug them in with pylons. I believe the reddit guys had a similar experience.

mrshoe2• 17 years ago

Exactly right. This is not an argument to not use Django.

I think a lot of us have come to the same conclusion as this author: the abstractions frameworks provide are often too high level. Very quickly the needs of your project cannot be served by their generalizations. Especially in a language like Python, it's often easier just to roll your own systems.

However, starting off with a framework is a great way to get off the ground quickly. If you were to write everything from scratch from the beginning, you might burn out before you even produce anything usable.

Personally, I think CherryPy is the way to go. You're probably not going to need to rewrite HTTP [1], so that's an abstraction layer with which I'm comfortable. Everything else will need to be easily customizable for your project.

[1] Unless the threaded model doesn't work for you, like it doesn't for our chat backend.

jp_sc2• 17 years ago

Have you tried Werkzeug (http://werkzeug.pocoo.org/)? It's a very interesting piece of technology. Like CheeryPy plus a Rails-like dispatcher.

jamesbritt• 17 years ago

"Exactly right. This is not an argument to not use Django."

No, but it might be an argument to consider frameworks that make it easier to replace parts as they become more trouble than they are worth.

mlLK• 17 years ago

It doesn't even sound like he made an effort at learning django, let alone python. . .look, django has a bit of a high learning curve behind it (I'm still at it myself) but when it comes down to its a web-framework, a web-framework for people who like programming in python.

I'll admit I felt the same way at first, but I was honest with myself enough to stop toiling with django and set aside the time to learn how to program in python. Once I started pin-pointing the obvious advantages of writing application-level programs in python I started to grasp the underlying advantages of django's design and philosophy.

pmclanahan• 17 years ago

That's the impression I got as well. He didn't seem to "get it." I can't see complaining about Django being poorly decoupled, only to then go on to describe how little of it you're still using. How tightly-coupled can it be if you're only using small pieces?

Also, it seems like a lot of the complaints are about contributed apps like authentication and the admin. It seems to me that he never really tried to learn the admin, or was using a very old version. And you don't have to use authentication at all, you can completely write your own, or fork the one that comes with it and make it work for you.

But the main thing that bugs me is that he obviously doesn't get opensource either. If you think you have a better way, get on the mailing list and suggest it, create a ticket, or implement your changes and submit a patch. Writing your own stuff, not releasing it, and complaining on a blog doesn't help anyone.

eculver• 17 years ago

If I could vote this comment up a thousand times I would.

snprbob8610• 17 years ago

The admin is bar far the biggest offender of tight coupling: primarily to authentication and the ORM. Being "contributed" isn't an excuse. Everything is "contributed". It's open source!

I haven't got time to actively participate in Django's development. Especially because I'm not particularly interested in the details discussed on the mailing list and I've already found components I'd prefer to work with (and, by extension, contribute to). I've got a product to build. I took some time to write down my thoughts and the key contributors are welcome to read my rant, ask questions, and learn from it.

mrshoe2• 17 years ago

While I agree with the premise of this comment, I'm not sure it necessarily applies to this specific situation.

Web frameworks do their best to provide useful, general abstractions for writing web applications. Sometimes you need different behavior on a lower level to which the framework provides no interface. In other words, your particular application breaks the general mold in some way. It doesn't necessarily make sense to change the framework for each of these application-specific needs. Writing those portions yourself and not contributing them to the project seems like a sensible way to go.

If the framework is really trying to work for the entire universe of web applications, it will quickly become too bloated to be useful for getting simple, common apps off the ground. As a framework maintainer, you have to strike a balance between the set of applications that your framework can accommodate and the usability of the framework.

That said, the author's tone does seem to suggest that Django is doing a poor job as a web framework because it doesn't fit his needs. That's an unfair criticism. Django fits a lot of people's needs. It's not supposed to be all things to all people.

ubernostrum4• 17 years ago

Well, with auth I think there's an easy confusion to fall into, because we conflate a couple things:

1. The default backend stores information through the User class, persisted to the auth_user table.

2. All backends are required to return instances of User when asked to find the user who matches a given set of credentials.

So in the first case, we use the User class as a persistent object store. In the second case we use it as a consistent API implemented by records coming out of auth backends.

Where the confusion comes in is people assuming that, since you have to return an instance of User out of your custom backend, you must also have to persist that instance to the auth_user table. Which isn't correct; you're free to just chuck information into an instance on the fly, and Django will neither know nor care (third-party code may make bad assumptions about being able to save that object, but third-party code should be introspecting to find out where the object came from before trying that).

Any rate, this is on my list of things to document more completely, and hopefully that'll help.

snprbob8610• 17 years ago

I know Python quite well; I have done several significant projects in it over the last few years. I have also read every page of the Django book and the vast majority of the official documentation. I became quite familiar with the internals of it, as I have spent many hours studying the source code before setting off to completely customize things.

What did I say which indicates a lack of experience?

I didn't feel this way in the beginning. At first, I quite liked Django. It was a huge time saver. I don't even really dislike Django now, our project has simply out grown it.

steamboiler• 17 years ago

"We are now using SqlAlchemy via schema reflection; no declarative layer." (emphasis mine)

What next? Will web development come full circle to using raw SQL queries like Phil Greenspun suggested years ago? (Not that there is anything wrong with it)

snprbob8610• 17 years ago

My next post will be about ORMs and the declartive paradigm in particular. It may prove to be more controversial than this post :-)

wehriam• 17 years ago

I wish. I find raw SQL much easier to use than any of the abstraction layers.

ams6110• 17 years ago

Nice to see at least one other person thinks so.

jp_sc2• 17 years ago

Web.py does something like that. Not raw SQL but almost:

    result = db.select('table', what='id, name', 
        where='id>3', order='id', limit=5)

Is better than cursors, but for medium to large web applications is too much work.

I used to be against ORMs ("Another layer? How inefficient"), but now I see them like compilers: They're easier to work with and, most of the time, SQLAlchemy does the job better than me (lazy-loading, etc.).

gecko2• 17 years ago

I've worked with a number of different ORMs, including two on the project I'm on right now, one custom one, and a handful of open-source ones on Ruby and Python and the like, and every single time, the story's the same.

At the start of the project, when queries are few and datasets are small, all of the above worked great. But the further you get into the project, and the more complex things you are trying to do with larger and larger datasets, the ORM ends up blowing up in your face. For example, LINQ to SQL--which I want to emphasize I'm picking on only because I'm using it righ tnow--happens to silently and transparently convert from IQueryable (meaning that your LINQ will execute in the DB) to IEnumerable (which means you'll suck in all data required for filtering past that point), which means that you never have a good idea whether your query is going to be optimized or not unless you check the SQL being generated. When you have to do this a few dozen times, you start wishing you'd just written SQL in the first case.

The situation gets worse when you start getting into join-heavy queries. I've never seen an ORM that actually can cover everything you need to make an app performant. It doesn't handle full joins, or compound indices, or distinguishing between clustered and nonclustered indices, or calculated columns, or some other thing that, if you were in raw SQL, would easily solve your problem, so at that point, you start introducing raw SQL. The older your app gets, the more instances of this you get, until you'd have been better off simply using SQL the entire time anyway.

I love the idea of an ORM in theory, but I'm pretty solidly convinced these days that if you want objects, you should us an OODBMS, and otherwise, you should just embrace the relational model for what it is, and not pretend you're working with objects. For most people, that'll mean writing SQL and getting back query sets.

snprbob8610• 17 years ago

I haven't used Linq to SQL except for to play with it for a few minutes. It seemed like a very capable ORM, but I sensed that the IQueriable to IEnumerable conversion may cause some headachs.

Linq to Objects, however, is freaking amazing. C#'s functional and querying features are like Python's generator expressions on crack, speed, steroids, and acid. Absolutely love them.

moe• 17 years ago

My bet would be schema-free databases, such as MongoDB.

Schema synchronization is just a major headache, no matter from which end you approach it (declarative -> evolution or Raw DDL -> reflection).

Once your number of databases exceeds 1 you are in trouble either way.

The implicit assumption that a schema definition would ideally be set in stone once and then never changed is a legacy carried over from the very early RDBMS days. And it's one of the biggest albatrosses to date.

lacker2• 17 years ago

I've had similar experiences... too often Django makes it difficult to do something really simple. If you are not building a newspaper-like site that fits exactly to the Django model, it gets annoying.

But it does support a lot of stuff out of the box, so it's still my favorite python web framework, and I have probably set up about a dozen different Django projects.

forsaken• 17 years ago

What problems did you exactly run into? I hear a lot of people say this, but never get good examples, which would hopefully make the framework better.

paulgb4• 17 years ago

The article gave some examples that I've run into myself: using email addresses instead of usernames for authentication, and adding additional fields to the User class. I also dislike the template language. Why invent your own language and control structures when everyone using it knows python? Mako (http://www.makotemplates.org/) does a good job of not reinventing the wheel.

lacker2• 17 years ago

One problem I often run into is that the Django libraries expect the whole application to be run by something that uses manage.py, which sets particular os.environ settings. So if you make code dependent on a Django library, often it cannot be run any more by non Django code. The eventual result is that things like cron jobs and unit testing are second-class citizens.

Another problem is unicode. Current Django assumes you will deal with all strings as valid unicode, which is actually different from pre-1.0 Django. There is no way to store an arbitrary sequence of bytes without a unicode encoding. This makes perfect sense for a newspaper website, but is problematic for many other applications.

Another problem is imports in general. Django does a lot of work in __init__.py files which causes problems integrating it with other systems that do a lot of things at run time.

ghostwheel• 17 years ago

And what project does this refer to? It might be more informative to see the actual app and then compare it to his comments.

snprbob8610• 17 years ago

I'm trying to keep my work, personal, and startup lives seperate. My project is no secret, but one of my recent blog posts was trafficed by some co-workers. I don't need my manager wondering if/when I'll be quiting.

Our site: http://www.classlet.com

Announced here: http://news.ycombinator.com/item?id=679000

There is a new version on the way with a huge host of new features and some significant cleanup. The version that is live now is more of a proof of concept.

tvon• 17 years ago

Django is not without it's problems, but some of this isn't correct (upon a quick skim). For example, the Django admin does support filtering and sorting, you just need to indicate this in the admin class.

snprbob8610• 17 years ago

You can set sorting order and filtering options in your admin class, but the UI doesn't provide clickable column headers or anything like a search box.

EDIT: Apparently I was mistaken, as the replies point out. I read the Django book, but I do not recall this chapter. My apologies.

jonknee• 17 years ago

Maybe I am misunderstanding you, but it does do clickable column headers (by default even I think) and you can definitely have a search box.

sebg• 17 years ago

I was going through the django book and here is something that could help you out:

search_fields = ('title',)

"Finally, the search_fields option creates a field that allows text searches. It allows searches by the title field (so you could type Django to show all books with “Django” in the title)."

http://www.djangobook.com/en/1.0/chapter06/

jbronn2• 17 years ago

The admin provides both of these features -- specify the columns you want to be clickable in the `list_display` attribute on your `ModelAdmin` class. Similarly, specify a `search_fields` attribute to add a search box for the columns you want to search.

Why read the documentation or look for examples when you can write a blog post screed and post it on HN?

intellectronica• 17 years ago

I'm no big fan of Django, but it sounds to me like you started using a well-established framework and whenever something was painful you chose to roll your own. Maybe a better approach would have been to engage with the Django community and help make these things less painful. That way your experience would benefit the next users, and you'd benefit from the collective wisdom and shared resources of a big and serious community. All other things being equal, rolling your own is rarely the right solution.

snprbob8610• 17 years ago

I only rolled my own URL routing. The complete implementation is less than 70 lines of Python.

Why am I obliged to engage the community when things are painful? I took an easier route: I choose less painful components.

dmoho• 17 years ago

I wouldn't say that you're obliged, but I'm sure they'd appreciate the feedback about what was painful and what wasn't.

Edit: I guess the article could be considered feedback :)

megaman821• 17 years ago

I don't quite understand the URL complaint. Each app can have its own urls file that you include in the main urls file. Isn't that a tree-like structure?

Also views can be served by callable classes, so an init function can populate the context (or a chain of them with class inheritance) and then the appropriate code can be run to serve the view.

snprbob8610• 17 years ago

It's a tree of lists of trees. I rather a tree, than tree-like :-)

Some of our apps have deeply nested paths (typically a no-no, but actually quite appropriate for our problem domain), so it is impractical to create a module and call "include" for each tree node.

As for callable classes: someone has to instantiate them. That's no different than our custom routing and authentication. I just choose function composition over class inheritence.

tptacek2• 17 years ago

"We wouldn't ever hire a designer who couldn't pass a code review for some trivial template logic."

I've worked with four design firms now, three of them excellent, and none of them could have been expected to understand templating code. If you're asking your designers to work with a template language, you're doing it wrong.

This has nothing to do with whether you put logic in your templates or not. The designer's deliverable isn't a working site; it's HTML/CSS/PNGs, and it's a dev's job to pull that into the framework.

po• 17 years ago

Those two opposing viewpoints on a designer's role probably have a lot to do with the type of application you are designing. If I was doing the hiring, I would expect more template language understanding for a web application than from marketing or traditional web site.

I think it's fair for him to say he wouldn't ever hire a designer like that but I think it's because he's only built sites where it makes sense to optimize for that.

tptacek2• 17 years ago

Why on earth would you want to narrow your options? There are dozens of template languages. Why would you want to spend one cent extra, or sacrifice one extra cent of quality, to find a designer who was not only skilled in information design, graphic design, typography, and interaction design, but also Django (or Erb or PHP or Clearsilver) literate?

snprbob8610• 17 years ago

If you're going to have external designers: I agree, their deliverables include ZERO logic.

However, zero logic is even less than Django templates. If it is the devs jobs to retrofit the HTML/CSS/PNGs with templating logic, it is an even bigger argument for choosing a developer friendly template language instead of a designer friendly one.

zmimon• 17 years ago

I've heard the refrain about keeping logic out of templates so many times, and yet professionaly I've never come across a situation where totally programming-clueless designers were actively editing templates. It always falls into one of two categories:

* the designer doesn't go anywhere near the real app and does all their work as mockups of HTML / CSS or even Illustrator / Photoshop. Developers do the work to integrate templates, designers at most fiddle with the CSS or bare HTML in the template and never touch any logic.

* the designer is reasonably competent to deal with the primitive programming in templates and is involved in the full lifecycle of product development

The bottom line is that if they are too leary of programming to cope with simple logic structures in a template then they are altogether incapable of working productively as part of the development lifecycle and should just hand their stuff to people who have those skills.

viraptor• 17 years ago

It's bizarre... I started writing a project in django 2 weeks ago and had to think about some of those elements in the first week. But I'm still running "clean" django!

Here are my solutions for anyone interested:

* URL Routing: They mention only breadcrumbs functionality here. I created breadcrumbs by template inheritance instead. There are url routers split by application and by modules. Template inheritance tree resembles the url routing levels, so that each template has: "{% block breadcrumbs %}{{ block.super }} · current-level-text{% endblock %}" Simple, but works so far :)

* Authorization: My views are decorated with @require_auth() if they require authorisation, @require_auth('group name') if they require special privileges and @require_auth(redir_to='...') if user should be redirected to a special place after authorisation. (by default they come back to the same place) Simple and works.

* Authentication: I simply didn't use the standard user classes. I used my own model. I don't consider it dropping Django - it's more like not using standard helpers.

* Templating: So far I'm happy with the whole logic in the view. Not sure how complex their application is... I'd really like to see some samples that required programming in the template...

* ORM and Admin UI: Can't comment on this one. I'm writing for google app engine and simply using their object store. It feels more comfortable than RDB so far. But sometimes I use GQL if it's faster.

* Form Validation and Generation: I've created a couple of dynamic forms extended with dynamic fields by jquery, as well as some forms with fields and validation generated on the fly (from user input, or saved data)... I can't find anything that's not possible with django forms. Sometimes I customise forms as needed, sometimes the autogenerated ones are almost perfect. Again - not using helpers all the time is not "dropping django".

I'm a bit disappointed that they didn't provide more examples and their solutions... maybe we could learn something from that.

paulgb4• 17 years ago

I've found the same thing with django. It does some things well (the admin interface, for example.) Other things it does in ways that seem very unintuitive and unpythonic.

For example:

- includes a test framework, but tests on a fresh project don't pass

- no relative paths in settings.py make it hard to work with other people (sure, since settings.py is python code you can write code to get around it, but I don't like that approach)

- you can't extend the User class (except by using deprecated code); the workaround is to create a one-to-one table but that's ugly.

- templates have their own control structures rather than using the ones pythonists already know (django's ifequals vs. python's if, for example)

If you're fed up with django, try Pylons. It is very loosely coupled, to the point where the recommended ORM and template system are separate projects (SQLAlchemy and Mako).

dnaquin• 17 years ago

I really should start making a list of things horribly broken in Django. By design.

I write off and hack around problems almost daily.

bad_user• 17 years ago

I also did my own URL routing ... http://code.google.com/p/django-catalyst/

I copied the chained actions model from Catalyst, and the routing is automatically generated. I also have template tags defined that construct these URLs for me (not uploaded in the repository).

I structure my views in a structure like ...

   views/root.py
   views/articles.py
   views/users.py

views.root contains a "begin" method, that will be chained to all other requests. If views.articles needs users to be authenticated, I also add a "begin" view to views.articles (where I do the authorization required). So as an example, a chain could look like this ...

   views.root.begin -> views.articles.begin -> views.articles.item -> views.articles.show

This is really far from the way things are done in Django, but it seems more intuitive to me.

What I find cool about Django is that the dispatching is really simple to understand, and it can be extended (and components aren't hard to replace in general).

Dropping Django

Related Stories