Background task processing and deferred execution in Django

http://www.turnkeylinux.org/blog/django-celery-rabbitmq

by liraz • 16 years ago

27 2 16 years ago

pxlpshr• 16 years ago

we use celery for task scheduling in the web app we're building. it's pretty fantastic for our needs, haven't had any major issues with it. they just announced 1.0 but we're still on .9.

patio11• 16 years ago

[One motivation for background task proessing is to g]ive the impression of a really snappy web application by finishing a request as soon as possible, even though a task is running in the background, then update the page incrementally using AJAX.

This is a spiritual cousin of progressive enhancement (an old technique from the days of 24kbps modems where you'd arrange a webpage such that the layout loaded quickly and then filled with those huge bloated 20kb Under Construction GIFs you were sending the browser), and while it increases complexity for the programmer, I think it has real possibilities in a wide range of web applications. If you look at the YSlow presentations, it is absolutely insane how much of a discrepancy there is between user-perceptible loading time caused by the back and and the front end, so if you get something to the front end as quickly as freaking possible, you get "free" time to continue crunching on the back end and update the skeletal response with minimal user-perceptible delay.

I get a lot of comments from my users that my site is SO FAST!!!1 because of this. The site really isn't fast -- it is just responsive, which is something that is easy to mistake for fast. For example, the quite misnamed "live preview" feature on one of the pages (which causes about 90% of my CPU load) creates a series of images which take ~2 seconds of CPU time each to spit out and pushes them to the user in response to user input, which makes it feel to most of my users like it is springy like a native application rather than a "Type a bunch into a form then hit submit, watch an hourglass for a while then a page load, and finally see stuff" that they expect on the web.

Background task processing and deferred execution in Django

Related Stories