Async Views in Django 3.1

https://testdriven.io/blog/django-async-views/

by rohitsohlot • 6 years ago

153 81 6 years ago

Writing asynchronous code gives you the ability to speed up your application [...]

Citation needed... It will be slower by definition but it probably advertised higher throughput anyway. I wonder how many people the throughput question is true for. You’d have to be pretty hardware constrained.

They even suggest asyncio could be a replacement for celery... It really is the Wild West out here.

toxik• 6 years ago

Yeah the post conflates asynchronous and concurrent like people always do. Asynchronous code can be executed sequentially, people. (Synchronous code can be executed concurrently, too.)

Asynchronous code is absolutely not just for running tasks after returning an HTTP response. A good example of the true power of asynchronous execution would be to do a Web Socket handler _inside a Django view._

fiedzia8• 6 years ago

They even suggest asyncio could be a replacement for celery

Only if you don't care about reliability.

nickjj• 6 years ago

They even suggest asyncio could be a replacement for celery... It really is the Wild West out here.

Yeah this is pretty questionable.

Having state in your app's process and then having to pretty much re-invent queues, job cancellation, uniqueness guarantees, scheduled jobs, periodic jobs on an interval, queue draining and retries with exponential backoffs.

I mean, yeah you could code all of that with enough time but why would you when Celery exists. If anything having the queue's state held in Redis or anywhere outside of your app's process is enough of a reason to pretty much never replace Celery with asyncio. Even in the post's example of sending an email I would still want a lot of what Celery has to offer and wouldn't consider replacing it.

IMO Celery isn't going anywhere soon and I would still use it in every Python project that needs background processing.

sbelskie2• 6 years ago

Just based on my C# experience, this sounds like a terrible idea (abandoning a real task queue for Async/await). 90% of the “gotchas” with async/await in C# comes from people thinking that the task scheduler is bonafide background queue.

mayama• 6 years ago

Same here. We do lot of processing in background jobs, which make no sense moving to async task.

earthboundkid2• 6 years ago

Celery doesn’t guarantee uniqueness. :-/

calpaterson5• 6 years ago

They even suggest asyncio could be a replacement for celery... It really is the Wild West out here.

Yes that is complete madness. The last thing I would ever want to do is send an email or use an external API (those are the examples in TFA) in an async thread as part of a request-response cycle. How does error handling or retries work for that background stuff? I don't know the exact ins-and-outs of Djangos new async stuff yet but my guess is that if that mail/API server doesn't respond to your IO then your response to your own HTTP request hangs?

Above all the central and undemonstrated claim is that there are any performance or management advantages to async. I tested sync vs async on this a couple of months ago. The best performance and robustness is with garden variety sync python with a good WSGI server:

http://calpaterson.com/async-python-is-not-faster.html

If anyone has any contradictory benchmarks please let me know. I haven't been made aware of any yet.

rahuldj• 6 years ago

Techempower benchmark: https://www.techempower.com/benchmarks/#section=data-r19&hw=... makes the difference between async python frameworks and sync frameworks pretty clear.

Async results in better utilization of system resources... since there aren't entire threads with app state blocked on IO requests (which would consume a chunk of system memory.)

Considering that a web app is most commonly waiting for a database operation, a single server can serve many more active requests simultaneously (since each active request is not allocating an app instance.)

collyw5• 6 years ago

I used Celery (as a workaround) for a Django view that should have been in asyncio before asynchio was available.

agumonkey• 6 years ago

somehow the idea reminds me of continuations based responses (HN used to be wired that way, also Queinnec wrote lisp papers on the topic)

Epskampie2• 6 years ago

I'm very familiar with symfony, but not so much with django or other "persistent in memory" web frameworks. Can someone ELI5 what the advantage is to async django? Does it mean the server doesn't block on one request before handling the next?

fiedzia8• 6 years ago

Yes. Typically to serve several requests at the same time, you'd run multiple processes, each handling an at a time, and if you spend a lot of time waiting for io (database, some service call or slow client), you'll need to use a lot of workers (each being independent python+django instance, so its a lot of memory -> significant investment in hardware). Async is nothing new in Python world, but previously you'd have to resort to pure python to use it, now you have whole django ecosystem that will use it. So main benefit is that you can handle more requests on the same django instance, and pay less for hardware.

Epskampie2• 6 years ago

Very clear explanation, thanks!

jordic8• 6 years ago

In our use case, from 12 cores to 2 :) (mostly for ha)

throwaway13337• 6 years ago

I recently moved some code base from py/django to c#/aspdotnetcore.

C# probably has some of the best async primitives of any language. I still managed to be bitten by the very common issue of error logging getting lost in a hidden callback that doesn't look like a callback because of the magic of the async keyword (a common issue in node as well).

I'm convinced that the async style of web servers is largely a step in the wrong direction. A leaky abstraction that we don't need.

A better model already exists - thread per connection - for the majority of what we want as web developers. There are some good things about allowing us to manage our own waits but those things are rare.

It reminds me of the choice of when to manage your own memory - usually we don't need to, but sometimes we do. Most languages programmers use today decided, rightly, that memory management was an exception and not the norm. We are more productive today for the reduction in cognitive load for most applications as we decide not to manage our own memory.

Thread per connection has gotten a bad name for performance which is pretty false. You can create the exact same underlying abstraction with green threads for more concurrency when you absolutely need it (but mostly you don't). The code we get from not using async is reduced in complexity substantially.

I'm disappointed that django is on the async bandwagon. It means that I may have to make another framework choice in the future.

GiorgioG• 6 years ago

As a C#/.NET developer I couldn't agree more - more often than not most of us simply do not need async, but everyone's doing it so it's become a 'best practice.'

merb• 6 years ago

well the problem is that if you use it once it's probably easier to use it everywhere. most people understand it if I show them a task explorer or ps and call Thread.Sleep and Task.Delay. I've never had an Issue with C# async and before I was a big scala user and did not have probems at all.

melolife• 6 years ago

The new async Razor engine in .NET Core is ridiculous. They threw out view helper functions and support for functional/monadic control flow for what, so you can asynchronously render a string? Insane.

I suspect the actual motivation is that async is virtually required to address performance problems associated with the new tag helpers - which are themselves a terrible, terrible idea. Slow, brittle, surprising, wholly reliant on syntax highlighting, and then redundant because you can implement the same functionality using the existing C# syntax.

collyw5• 6 years ago

As someone who has been in the industry for 20 years everything seems to be getting worse. Like Kubernetes and SPA apps. People rarely need these (OK, high availability via Kubernetes is nice, but few people need elastic scaling).

tester756• 6 years ago

I still managed to be bitten by the very common issue of error logging getting lost in a hidden callback that doesn't look like a callback because of the magic of the async keyword (a common issue in node as well).

What do you mean, any example? I have code full of asyncs and never had problem with logging

sbelskie2• 6 years ago

The biggest culprit of this kind of thing is usually people doing something with Task.Run thinking it’s a free way to queue a background task.

nemothekid2• 6 years ago

Anything that spawns an unbounded amount of POSIX threads will be bad for performance, so you need an M:N scheduler to execute threads. I feel like this is pretty hard to retrofit onto an existing language (e.g. I don't think you can write a pre-empting scheduler in pure Python as a library either).

AFAIK, only Go does this well (with great results) as a result of it being pretty much baked into the language. I don't think it's an async bandwagon, I think async/await is the best way to get concurrency without requiring the whole language to opt-in (as the Rust devs discovered).

thomasfortes• 6 years ago

Elixir does that too (more a BEAM thing to be more precise), the BEAM can spawn millions of processes without killing your server performance.

But I agree with your main point, concurrency is hard if the language isn't built from the ground to be concurrent.

YorickPeterse2• 6 years ago

I think it's not necessarily POSIX threads related. In theory there is nothing wrong with spawning tens/hundreds of thousands OS threads, and Linux does a decent job at this. But in practise you'll run into issues such as:

* Different OS' handle things differently. macOS limits the number of threads per process (somewhere around 1500 I believe). Other OS' may be slower, or impose other limits

* Because of this, sometimes spawning threads is pretty fast (e.g. 10 µsec). Other times it's super slow (I've seen threads take over 500 ms to spawn)

* Context switches are still pretty slow across OS'. There's finally some work being done towards lightweight thread support in Linux, but I suspect it will take a few years to become useful

I _really_ hope that one day we _can_ "just" spawn 100 000 OS threads without issues, as it would make many concurrency problems easier to solve. Sadly, I think it will take another 10-15 years.

winrid• 6 years ago

Not as bad as you think, for an application server anyway. HN actually uses a thread per request according to the docs: https://arclanguage.github.io/ref/srv.html

They also read from disk for most requests. So, thread per request and also reading from disk.

I worked at one company where we had many dozens of services with thrift/spring boot and spawned a request per thread and it worked fairly well. We were collecting and aggregating every review on the internet (for our customers), and had dozens of services... There were a couple in a reactive style. So we took the maintenance cost when we needed to.

dtech2• 6 years ago

I get where you're coming from, but I disagree.

Async code has its traps, but they are very managable once you've learned them. I haven't found them on the level of manual memory management which is a footgun that even experienced devs keep firing.

The benefits are also there, doing slow things in parallel like HTTP calls or database queries is trivial with async code and trickier in sync code.

Of course you prefer sync code when you don't need it, but async/await style code doesn't prevent sync code. If you only support sync doing anything async because very cumbersome and dangerous, and doing so requires things like managing threads or locks yourself which is an even greater recipe for bugs.

calpaterson5• 6 years ago

You can of course do parallel HTTP calls or database queries with Python's existing threads (IO releases the GIL in Python) or via multiprocessing.

You don't have to use locks either - straight parallelism can be done very easily with the Executors in concurrent.futures. There are a lot of good concurrency primitives in the threading library in the stdlib including conditions, semaphores and barriers. Of course best to avoid parallelism with shared data if at all possible (and IMO it doesn't really come up much).

zerkten• 6 years ago

Async has a place, but as you say there may be a better option for the majority of developers. The problem is that many developers are not identifying their performance issues and instead just throwing solutions out there. Perf discussions with plain Python often devolve into a GIL discussion before anyone confirms if the problem is CPU-bound.

I see many issues from developers where I have to walk them back to the start and spend time on identifying the problem. There are many reasons for this, but it's not necessarily the most fun or exciting thing for developers to do. There are still many artificial impediments because devs don't universally have the control over, or access to prod environments that startups may have. If this troubleshooting is a hurdle then sprinkling in some async starts to seem much simpler.

quadrifoliate• 6 years ago

I see many issues from developers where I have to walk them back to the start and spend time on identifying the problem.

Honestly, in a lot of cases when I have thrown away something and used something else because I just need to show that I'm doing something to develop a reasonably performant application rather than spending all my time optimizing performance. Spending a whole week optimizing your application's performance is surprisingly rare in the modern hustle-driven culture of "lean" startups and such -- cargo-culted by a lot of companies and bosses who have nothing to do with startups.

I recently realized that the entire gunicorn / uwsgi world for example is filled with anecdotes about performance instead of simple, comprehensible benchmarks; and I thought they were one of the better ones. I suspect this kind of lack of performance focus by most of the users is a big part of why.

simonw• 6 years ago

Django agrees with you. The async design for Django is predicated on the idea that the vast majority of cases are better handled by threads - but very occasionally it is useful to run an async view for something that's very IO bound (calling other HTTP APIs most likely).

If you watch Andrew's talk from PyCon AU last week he explicitly calls out that sync views are still better for most cases, which is why he invested so much effort in making sure they were not negatively affected by the addition of async: https://youtu.be/ibAmA4QQDhs

theptip2• 6 years ago

This is something that lots of folks seem to be getting confused about, with benchmark posts coming up on HN asking “is Django async faster than X?”.

For the API gateway use case this feature is really useful, and extends the usefulness of Django by letting it serve as a gateway for longer as your app grows and you start extracting services from the monolith.

I think part of it is the idea that in principle calling your DB is IO - but in practice your DB is fast enough that your sync view is not spending much of its time waiting on that IO.

devxpy2• 6 years ago

But here's the thing. In all my encounters with asyncio, its slower, even for things like HTTP calls

I suspect its because the event loop gets blocked doing trivial things like json/csv parsing, but it could just be python's slowness.

Asyncio seems super hard to profile too..

fiedzia8• 6 years ago

A better model already exists - thread per connection

For that you will need a language with good support for threads, which Python definitely isn't (at least not CPython), so Django doesn't have any other option here.

Native threads are huge improvement over async, but still not as good as green threads, which really are the best option here for 99% of usecases, abstracting whole async thing away.

Thread per connection has gotten a bad name for performance which is pretty false.

With 64 or more cores being the norm, a thread with 2MB stack per connection requires 128MB just to exist (though I believe OS may offer some magick if you are not actually using it). This is probably 3-4x time what's required for green threads.

Scarbutt• 6 years ago

With 64 or more cores being the norm, a thread with 2MB stack per connection requires 128MB just to exist

Sounds good to me!

Can you even find a VM offering that has a 64 core CPU and less than 32GB of RAM?

Something1234• 6 years ago

>A better model already exists - thread per connection
For that you will need a language with good support for threads, which Python definitely isn't (at least not CPython), so Django doesn't have any other option here

The thread per connection is managed by your application server. Why does the language need to have good thread primitives then? In fact I would argue you spawning and managing your own threads make your service less robust.

dagenix• 6 years ago

With 64 or more cores being the norm, a thread with 2MB stack per connection requires 128MB just to exist (though I believe OS may offer some magick if you are not actually using it). This is probably 3-4x time what's required for green threads.

Are you assuming that each thread requires a dedicated stack on each core? That isn't how it works - each thread has a single stack regardless of the number of CPU cores. So, a 2MB stack requires just 2MB. However, that 2MB is 2MB of virtual memory. So, assuming that the stack isn't allocated using a 2MB huge page, it will probably actually be physically allocated in 4K chunks as required. So, most threads are going to use considerably less than 2MB of actual memory. Outside of 32bit systems, virtual memory is close to an unlimited resource.

Green threads also require stacks. How green threads allocate stack space is going to depend on the particular runtime, but it's likely that like with real threads they will also lazily allocate memory as required. A big difference between stack usage with green threads and regular threads is that the runtime _may_ support shrinking the stack used by a green thread if space is only briefly required by a green thread, while a regular thread will hold on to any memory that is physically allocated for its stack until it exits. Reclaiming memory like this isn't free, however - and you run the risk of "stack thrashing" where the point where a stack grows and shrinks is accidentally in the middle of a hot loop and you end up spending basically all your time growing and shrinking the stack as opposed to doing real work.

I _think_ that some green thread runtimes may also support stacks smaller than 4K - but I'm not really sure if / how many actually do.

nurettin• 6 years ago

async views aren't there because django is moving in that direction, they are there to make it easy to use async functions that already exist. I write a lot of functions that block on IO and a lot of them benefit from async event loops. Now I have a simple way of integrating that functionality into django views.

lazyant• 6 years ago

A better model already exists - thread per connection - for the majority of what we want as web developers.

Haven't we already seen in Apache vs Nginx that thread per connection will OOM the web servers under heavy traffic? (haven't used Apache in many years so this may be obsolete)

drzaiusx11• 6 years ago

Apache moved from using a thread per connection to fibers years ago [1]. Fibers are simpler than green threads (cooperative multitasking via yielding) which should be illustrative of how good an idea the "thread per connection" model is.

Although it could be argued that fibers are a type of primitive thread--so as long you don't mean OS thread the idea has merit.

[1] http://httpd.apache.org/dev/whiteboard/process-model.html

nesarkvechnep• 6 years ago

OS thread per connection? Nah. BEAM process per connection - hell yeah!

collyw5• 6 years ago

When you say "threads per connection" do you mean "process per connection"? There isn't a lot shared between the connection in the way that I think of threads.

Anyway, I agree that async seems to be more trouble than it's worth for most cases.

devxpy2• 6 years ago

This. I really agree with this.

The async afficianados also bring along the async ecosystem split.

I wish they did a better job of bridging this gap.

Languages like rust, which already have amazing support for threads don't need this split, please.

zests• 6 years ago

I've done asynchronous programming with the reactive paradigm where I specify a chain of function calls I want applied to data and it happens in a thread independent way. I have not done much programming with the new fancy "async/await" keywords. It seems like a worse abstraction because it mixes asynchronous code with synchronous code and is probably harder to reason about. Not to mention you can't do something like application wide backpressure unless everything is asynchronous.

Can anyone who has worked in both types of applications comment on their differences?

calpaterson5• 6 years ago

I would say that there are maybe three different ways of async rather than two. Broadly:

- wholly implicitly (by monkeypatching stuff with gevent typically)

- async/await

- by reactor (twisted)

async/await is not really an alternative to the explicit style of twisted. To paraphase Guy Steele it is an attempt to drag the monkeypatchers halfway to twisted. It doesn't really solve the problem of mixing sync code in with your async code, the only preventative to that is programmer diligence. If you read from a file using open in the middle of your async code you're going to block the process while bytes are gathered for you. However it does prevent you from accidentally using async code in the middle of your sync code.

Problems like backpressure and failure handling are largely unsolved in asyncio.

dtech2• 6 years ago

I've done both, but 80% in reactive streams. I like async/await before it has a lot less hard stuff to think about - hot and cold streams take months if not years for devs to grok - and creates code a lot more similar to non-async code. In contrast Reactive Streams requires you to change your complete program.

I like to compare it to Typescript/Kotlin-style null-safety v.s. the Java Optional or Scala Option. If you want make code null safe with option you have to rewrite basically your complete code from "procedural" to chains of (flat)maps. In contrast in Typescript you just change T to T? in a few places and add an if where you want to handle the optionality.

async/await versus reactive streams has this same problem, reactive streams is highly impactful, much more than async/await. In practice I've seen this have the effect of turning applications turn into a small reactive stream outer shell with the app itself being mostly synchronouse/Future, while async/await is used a lot more liberally because the impact is smaller.

It seems like a worse abstraction because it mixes asynchronous code with synchronous code

It doesn't, at least in Kotlin only an async function can call an async function. This isn't different from using `map` in a stream.

lmeyerov• 6 years ago

As someone who writes + maintains systems with both: They both have their place. Reactive streams (FRP, Rx, ...) are good when you want to do operations over streams like 'fold', 'sample', 'tween', etc., but most code isn't like that. Instead, most code is 'at-most-once' (=> value or exn), where async/await is enough. That matters because adding streams means all of a sudden your code looks like 'one-or-more', which adds an entirely new world of behavior to define & test against. ("Is this a value or a stream? If I feed in a stream... What if I just want one...")

'Behaviors' in FRP help a bit here... but if you can avoid the complexity, why not?

For when we do FRP, I like how we got it in our React code: Mostly synchronous functional code (React::render(), ...), and then we do animations and other stream-like concepts via Rx. Our nodejs code works similarly: mostly async/await route handlers, and now and then we do fancier http streams and whatnot via Rx. (Though even there async/await has been increasingly nice, as we generally aren't doing many fancy temporal operators nor contention handling.) We haven't figured out an equivalent solution for our Python async code, however as async/await is still new & changing, and afaict FRP/Rx never really made it there.

holler4• 6 years ago

For anyone looking for an alternative python async web framework, I highly recommend Starlette (https://github.com/encode/starlette).

It was created by the same guy who wrote the excellent & highly popular Django-Rest-Framework. Very lightweight, clean, well-documented, and a breeze to get running with. It also supports async tasks natively.

I spent years working with Django and now I'm currently using it to power the api for https://sqwok.im.

jordic8• 6 years ago

We are also using starlette and works quite well.. in combination with asyncpg..

holler4• 6 years ago

very nice! cool animations on tmpo.io! Right now I'm running things behind aws lambda & using dynamodb etc, have looked into using aioboto3 but haven't got there yet...

toinbis• 6 years ago

Do you use any db connection wrappers around asyncpg? Like encode/databases or gino? Also important to note that SQLAlchemy 1.4 alpha with asyncpg support is realease and can be tried out(both SQLAlchemy core&orm modes support it). I myself evaluated all options of async postgress connection wrappers in fastapi project but reverted to... just using sync psycopg2 through sqla core. No very specific gains of async db connection for my use cases and sqlalchemy creator's comments on immaturity of async postgres drivers made my stick to what's battletested.

scrollaway• 6 years ago

Have you tried fastapi? It uses starlette as its base. Such a lovely experience working with it.

holler4• 6 years ago

I haven't tried it but I did look it over when I was investigating Starlette. May try it out in the future, it seems powerful, and I read about a few projects using it. I ended up just rolling with Starlette because I liked how minimal, yet flexible it was (why FastApi etc built on it).

collyw5• 6 years ago

Some reasons I see for sticking with Django is that it is big and well known. So easier to hire devs that know it, plenty of knowledge on Stack Overflow and elsewhere. Plus a large ecosystem of packages. (We use Tornado at my current workplace, and we would be far better off with something like Django).

holler4• 6 years ago

Definitely, agree, Django is a great framework and I've used it over the past 5+ years at many companies large and small. When I set off to build my own project I wanted to keep my code as lean as possible and that's one thing I like about Starlette. It gives you the basic blocks and lets you decide how to use it.

jordic8• 6 years ago

With async Django views we can use Django with asyncpg... o_o but middlewares are still synch...

zaro• 6 years ago

Yeah, it will be some more years before you can simply write async stuff in Django and not worry about every single library that you use, whether it supports async or not.

That's why some years ago I gave up on Django altogether and moved to Nodejs. Much better experience writing async code.

jordic8• 6 years ago

Our front service it's a nextjs/nodejs project. But I still prefer python :) and our API's are python based, typed async python.. but python..

kabacha• 6 years ago

I really don't get this anti-async mentality on HN. Maybe I'm doing something wrong but async is a complete game changer for me as there just so much IO wait in every project I touch. The fact that my program has to stop and wait for something is honestly unacceptable and it's just a logical conclusion to eliminate this blockage.

I worked past 2 years with asyncio and I have to say it still kinda sucks. The syntax is overly verbose and asyncio itself can't decide what it wants to be. However it still works _good_ for the most part and there's an enormous community and effort in this!

BozeWolf• 6 years ago

“So much io wait”. What io is your app waiting for? Is it a webserver, or another type of app?

naters• 6 years ago

Not the GP, but I'm currently working on a small web app that synchronizes data from 2 other web services, which means that almost every endpoint involves a call to an external API. In some cases, it involves repeated calls to those APIs, but the order in which the responses are received is unimportant, which is an ideal use case for asyncio's gather function. Using a framework (I haven't tried Django's new async views yet, but am using one of the handful of Flask-ish async frameworks) that supports async views has made this a snap, whereas before I would probably have used some sort of out-of-process work queue type setup.

esseti• 6 years ago

is there a list of real use cases where the async is better than the sync. it seems that it's ok in very edge casees where there's high load and a lot of I/O (example, calling external APIs, calling a db).

however, is it stable enough and roboust to replace sync methods in production for the benefit of these cases?

Async Views in Django 3.1

Related Stories