A Visual Journey Through Async Rust

https://github.com/alexpusch/rust-magic-patterns/blob/master/visual-journey-through-async-rust/Readme.md

by PaulHoule • 1 year ago

151 25 1 year ago

OskarS• 1 year ago

Very neat visualizations! Really good way to demonstrate the underlying principles. Great article.

I will add one thing: the author makes much of how problematic CPU-intensive tasks are. That's true, but also: if you need to do a CPU-intensive operation, you need to tie up a CPU to do it, there's no getting around it. What you might want to do is break the computation up in to smaller chunks with `.await`s sprinkled in there or dispatch it to a background thread, but at some point you have to pay the cost in CPU time.

What is much more problematic is blocking I/O operations. That also ties up a CPU and blocks the event loop, but for no good reason. All the CPU is going to be doing is sleeping waiting for the network or hard-drives or whatever it is to do it's work, when it could be doing other useful things, processing other events. That kind of thing leads to huge under-utilization of your resources, and it kills your concurrency.

This means you really can't mix async I/O with blocking I/O: if you go async, ALL your I/O operations need to be async. Or you have to do it off the event loop in a background thread or something, but then you're losing all the benefits of using async in the first place.

remram3• 1 year ago

blocking I/O operations. That also ties up a CPU and blocks the event loop

They do block the event loop (if single-threaded) but they do not "tie up" a CPU (core), the OS will schedule other threads on it.

binary1322• 1 year ago

assuming you have other threads

but in an async I/O server scenario, the typical architecture is a fixed-size pool of worker threads, which ideally entirely occupy a single core each. therefore, if one of those threads is blocked, that core is wasted.

PaulHoule• 1 year ago

I went through a phase of writing a lot of little things in async Python which was a lot of fun but I wrote an app that really tied up the event loop and now, at least for web servers, I develop with Flask, deploy to gunicorn and often front with IIS or nginx to serve up images and big files now.

One thing I still like async Python for is programs that have a number of polling tasks going on at a slow cadence, say once a minute or less, where it doesn't really matter to me if a task gets delayed 5 seconds because of another task. In that case writing a number of tasks that await.sleep is easy and fun and works great -- I'll use async io in the tasks if it is easy to do so but if I use blocking I/O and it blocks... no problem.

At work I've been writing web servers in Java (or C#) for a couple of jobs and between threads and the garbage collector I also think it is easy and fun. Sure if I was handling Google-levels of traffic I might think differently, but threads scale pretty well for 99% of the web sites out there. Java gives you all the right primitives to do common tasks: sometimes I code something up with an Executor in 20 minutes and it works right the first time and gets a 14x speedup with 16 cores, somebody else tries Actors in the language of the day and spends a few days on it, never gets more than a 3x speedup and gets a different answer every time. (Threaded programming is easy when you have a rich primitive library you get in trouble when you get it in your head that it is noble to build everything in scratch from a small set of primitives)

Java even has virtual threads that play well with real threads if real threads aren't good enough for you.

vermilingua• 1 year ago

It’s never addressed in the article that the calculations running on spawned threads produce deformed sine curves. Is that just a result of how its plotted or do tokio threads have less accurate track of time?

alphaXp• 1 year ago

Thats a good observation. I believe the deformation comes from the variable runtime of the sin method. Even with the 100 micros sleep that I've put there, in reality it takes a different amount of time between each data point calculation, making the sine wave a bit wonky. This gets worse as more and more calculations are done in parallel

ninkendo2• 1 year ago

It's more likely to be the unaccuracy of Instant in rust. It's guaranteed to be non-decreasing, but not steady: https://doc.rust-lang.org/std/time/struct.Instant.html

In other words, each tick of the underlying clock might not be the same length (e.g. some seconds may be longer than others). An instant may jump forwards or experience time dilation (slow down or speed up), but it will never go backwards

LoganDark• 1 year ago

I think it's just scheduling. The deformation is when it didn't get to run exactly in time because Tokio had to wake up enough other tasks first.

jayd16• 1 year ago

Looks like it's just plotting a fixed number of points at the time the iteration was scheduled. It's not trying to map a smooth curve.

inglor2• 1 year ago

Unlike Node.js, Rust's Tokio allows us to spawn a new Task and run futures within it.

Nice article! For future reference in Node.js this would be `worker_threads` or if you want a higher level tokio like API something like https://piscinajs.dev/

This has been possible in Node.js for quite a while

Also for blocking I/O CPU bound parallelism I've found Rayon to be a pleasure. It has a few caveats but it leverages the Rust race condition checking well and it's very ergonomic.

I happily mix it with `tokio` in an application that is CPU bound but has to perform some async I/O (due to the SDKs it uses)

remram3• 1 year ago

A tokio task is not a thread

inglor2• 1 year ago

A node worker_thread is not a thread either though :]

SkiFire13• 1 year ago

piscinajs

Lmao this wordplay made my day

Rust race condition checking

Nit: Rust does not prevent race conditions, it prevents data races.

rc00• 1 year ago

Shared 12 days ago: https://news.ycombinator.com/item?id=43675098

binary1322• 1 year ago

This is not limited only to Rust and I'd be very curious to see a similar comparison across multiple runtimes!

jwufasdfsds• 1 year ago

The async disaster can be avoided by using Plan9 or Inferno.

Don't misinterpret this as criticism of async-rust. It's required to work on legacy platforms like linux.

whytevuhuni• 1 year ago

This is obviously a troll comment, but it made me curious.

I'm familiar with Plan 9, and I know about the interesting mount namespace shenanigans you can pull off with it, but does it actually have anything that would help with async?

jdnier• 1 year ago

Alright! We managed to plot two sine waves in the most convoluted method ever!

Those sine waves start to resemble a multi-voice music score. You might get some interesting pitch variability depending on other processes that are running.

ModernMech2• 1 year ago

"Visualizing futures runtime like this really makes some pennies drop."

What's it mean to drop pennies?

masfuerte• 1 year ago

It's a British idiom. When the penny drops you suddenly understand something.

https://en.wiktionary.org/wiki/the_penny_drops

tialaramex• 1 year ago

I was kinda hoping for a origin story, but alas this doesn't seem to lead to one, and then I checked an actual physical paper dictionary (I'm an old man, old enough that I bought a dictionary in 1982 apparently) and that does define this phrase but doesn't explain where it came from.

I have used this metaphor my whole life without thinking about why pennies dropping have this significance. So now I'm wondering

My best guess is that it's thinking about mechanisms which operate when given money, a topical example would be the Alzara fortune telling machine in a Rumpus Room of Blue Prince. If we put a coin in the machine, it springs into life and tells us... something. If you won't give it a coin, nothing for you.

A Visual Journey Through Async Rust

Related Stories