Why choose async/await over threads?

by thunderbong • 2 years ago

430 458 2 years ago

newpavlov9• 2 years ago

I think a better question is "why choose async/await over fibers?". Yes, I know that Rust had green threads in the pre-1.0 days and it was intentionally removed, but there are different approaches for implementing fiber-based concurrency, including those which do not require a fat runtime built-in into the language.

If I understand the article correctly, it mostly lauds the ability to drop futures at any moment. Yes, you can not do a similar thing with threads for obvious reasons (well, technically, you can, but it's extremely unsafe). But this ability comes at a HUGE cost. Not only you can not use stack-based arrays with completion-based executors like io-uring and execute sub-tasks on different executor threads, but it also introduces certain subtle footguns and reliability issues (e.g. see [0]), which become very unpleasant surprises after writing sync Rust.

My opinion is that cancellation of tasks fundamentally should be cooperative and uncooperative cancellation is more of a misfeature, which is convenient at the surface level, but has deep issues underneath.

Also, praising composability of async/await sounds... strange. Its viral nature makes it anything but composable (with the current version of Rust without a proper effect system). For example, try to use async closure with map methods from std. What about using the standard io::Read/Write traits?

[0] https://smallcultfollowing.com/babysteps/blog/2022/06/13/asy...

lmm6• 2 years ago

How do fibers solve your cancellation problem? Aren't they more or less equivalent?

(I find fiber-based code hard to follow because you're effectively forced to reason operationally. Keeping track of in-progress threads in your head is much harder than keeping track of to-be-completed values, at least for me)

bheadmaster6• 2 years ago

Keeping track of in-progress threads in your head is much harder than keeping track of to-be-completed values, at least for me

I think that's true for everybody. Our minds barely handle state for sequential code - the explosion of complexity of multiple state-modifying threads is almost impossible to follow.

There are ways to convert "keeping track of in-progress threads" to "keeping track or to-be-completed" values - in particular, Go uses channels as a communication mechanism which explicitly does the latter, while abstracting away the former.

newpavlov9• 2 years ago

With fibers you send cancellation signal to a task, then on next IO operation (or more generally yield) it will get cancellation error code with ability to get true result of IO operation, if there is any. Note that it does not mean that the task will sleep until the IO operation gets completed, cancellation signal causes any ongoing IO to "complete" immediately if it's possible (e.g. IIRC disk IO can not be cancelled).

It then becomes responsibility of the task to handle this signal. It may either finish immediately (e.g. by bubbling the "cancellation" error), finish some critical section before that and do some cleanup IO, or it may even outright ignore the signal.

With futures you just drop the task's future (i.e. its persistent stack) maybe with some synchronous cleanup and that's it, you don't give the task a chance to say a word in its cancellation. Hypothetical async Drop could help here (though you would have to rely on async drop guards extensively instead of processing "cancellation errors"), but adding it to Rust is far from easy and AFAIK there are certain fundamental issues with it.

With io-uring sending cancellation signals is quite straightforward (though you need to account for different possibilities, such as task being currently executed on a separate executor thread, or its CQE being already in completion queue), but with epoll, unfortunately, it's... less pleasant.

lloeki2• 2 years ago

I think a better question is "why choose async/await over fibers?
there are different approaches for implementing fiber-based concurrency, including those which do not require a fat runtime built-in into the language

This keynote below is Ruby so the thread/GVL situation is different than for Rust, but is that the kind of thing you mean?

https://m.youtube.com/watch?v=qKQcUDEo-ZI

I think it makes a good case that async/await is infectious and awkward, and fibers (at least as implemented in Ruby) is quite simply a better paradigm.

newpavlov9• 2 years ago

I only scrolled the video, but it sounds similar, yes. Though, implementation details would probably vary significantly, since Rust is a lower-level language.

dwaite3• 2 years ago

I think it makes a good case that async/await is infectious and awkward, and fibers (at least as implemented in Ruby) is quite simply a better paradigm.

As someone who has done fibers development in Ruby, I disagree.

CRuby has the disadvantage of a global interpreter lock. This means that parallelism can only be achieved via multiple processes. This is not the case in Rust, where have access to do true parallelism in a single process.

Second, this talk is not arguing for use of fibers as much as it is arguing as using fibers to rig up a bespoke green threads-like system for a specific web application server, and advocating for ruby runtime features to make the code burden on them of doing this lighter.

Ruby has a global interpreter lock, so even though it uses native threads only one of them can be executing ruby code at a time. Fibers have native stacks, so they have all the resource requirements of a thread sans context switching - but the limitations from the GIL actually mean you aren't _saving_ context switching by structuring your code to use fibers in typical (non "hello world" web server) usage.

logicchains6• 2 years ago

The sole advantage of async/await over fibers is the possibility to achieve ultra-low-latency via the compiler converting the async/await into a state machine. This is important for Rust as a systems language, but if you don't need ultra-low-latency then something with a CSP model built on fibers, like Goroutines or the new Java coroutines, is much easier to reason about.

newpavlov9• 2 years ago

Fibers and async/await are backed by the same OS APIs, they can achieve more or less the same latency. The main advantage of async/await (or to be more precise stackless coroutines) is that they require less memory for task stacks since tasks can reuse executor's stack for non-persistent part of their stack (i.e. stack variables which do not cross yield points). It has very little to do with latency. At most you can argue that executor's stack stays in CPU cache, which reduces amount of cache misses a bit.

Stackless coroutines also make it easier to use parent's stack for its children stacks. But IMO it's only because compilers currently do not have tools to communicate maximum stack usage bound of functions to programming languages.

pron3• 2 years ago

The key requirement for ultra-low-latency software is minimising/eliminating dynamic memory allocation

First, that is only true in languages -- like C++ or Rust -- where dynamic memory allocation (and deallocation) is relatively costly. In a language like Java, the cost of heap allocation is comparable to stack allocation (it's a pointer bump).

Second, in the most common case of writing high throughput servers, the performance comes from Little's law and depends on having a large number of threads/coroutines. That means that all the data required for the concurrent tasks cannot fit in the CPU cache, and so switching in a task incurs a cache-miss, and so cannot be too low-latency.

The only use-cases where avoiding memory allocation could be useful and achieving very low latency is possible are when the number of threads/coroutines is very small, e.g. generators.

The questions, then, are which use-case you pick to guide the design, servers or generators, and what the costs of memory management are in your language.

dwaite3• 2 years ago

For rust, fibers (as a user-space, cooperative concurrency abstraction) would mandate a lot of design choices, such as whether stacks should be implemented using spaghetti stacks or require some sort of process-level memory mapping library, or even if they were just limited to a fixed size stack.

All three of these approaches would cause issues when interacting with code in another language with a different ABI. It can get really complicated, for example, when C code gets called from one fiber and wants to then resume another.

One of the benefits of async/await is the 'await' keyword itself. The explicit wait-points give you the ability to actually reason about the interactions of a concurrent program.

Yielding fibers are a bit like the 'goto' of the concurrency world - whenever you call a method, you don't know if as a side effect it may cause your processing to pause, and if when it continues the state of the world has changed. The need to be defensive when interfacing with the outside world means fibers tend to be better for tasks which run in isolation and communicate by completion.

Green threads, fibers and coroutines all share the same set of problems here, but really user space cooperative concurrency is just shuffling papers on a desk in terms of solving the hard parts of concurrency. Rust async/await leaves things more explicit, but as a result doesn't hide certain side effects other mechanisms do.

newpavlov9• 2 years ago

In my opinion, by default fibers should use "full" stacks, i.e. a reasonable amount of unpopulated memory pages (e.g. 2 MiB) with guard page. Effectively, the same stack which we use for threads. It should eliminate all issues about interfacing with external code. But it obviously has performance implications, especially for very small tasks.

Further, on top of this we can then develop spawning tasks which would use parent's stack. It would require certain language development to allow computing maximum stack usage bound of functions. Obviously, such computation would mean that programmers have to take additional restrictions on their code (such as disallowing recursion, alloca, and calling external functions without attributing stack usage), but compilers already routinely compute stack usage of functions, so for pure Rust code it should be doable.

It can get really complicated, for example, when C code gets called from one fiber and wants to then resume another.

It's a weird example. How would a C library know about fiber runtime used in Rust?

Yielding fibers are a bit like the 'goto' of the concurrency world - whenever you call a method, you don't know if as a side effect it may cause your processing to pause, and if when it continues the state of the world has changed.

I find this argument funny. Why don't you have the same issue with preemptive multitasking? We live with exactly this "issue" in the threading world just fine. Even worse, we can not even rely on "critical sections", thread's execution can be preempted at ANY moment.

As for `await` keyword, in almost all cases I find it nothing more than a visual noise. It does not provide any practically useful information for programmer. How often did you wonder when writing threading-based code about whether function does any IO or not?

pron3• 2 years ago

you don't know if as a side effect it may cause your processing to pause, and if when it continues the state of the world has changed

This may be true in JS (or Haskell), but not in Rust, where you already have multithreading (and unrestricted side-effects), and so other code may always be interleaved. So this argument is irrelevant in languages that offer both async/await and threads.

Furthermore, the argument is weak to begin with because the difference is merely in the default choice. With threads, the default is that interleaving may happen anywhere unless excluded, while with async/await it's the other way around. The threading approach is more composable, maintainable, and safer, because the important property is that of non -interference, which threads state explicitly. Any subroutine specifies where it does not tolerate contention regardless of other subroutine it calls. In the async/await model, adding a yield point to a previously non-yielding subroutine requires examining the assumptions of its callers. Threads' default -- of requiring an explicit statement of the desired property of interference is the better one.

gpderetta16• 2 years ago

I never fully understood the FFI issue. When calling an not-known coroutine safe FFI function you would switch form the coroutine stack to the original thread stack and back. This need not be more expensive than a couple of instructions on the way in and out.

Interestingly, reenabling frame pointers was in the news recently, which would add a similar amount of overhead to every function call. That was considered a more than acceptable tradeoff.

cryptonector10• 2 years ago

All three of these approaches would cause issues when interacting with code in another language with a different ABI. It can get really complicated, for example, when C code gets called from one fiber and wants to then resume another.

Java (in OpenJDK 21) is doing it. To be fair, Java had no other choice because there is so much sequential code written in Java, but also the Java language and bytecode compilation makes it easy to implement spaghetti stacks transparently. Given those two things it's obviously a good idea to go with green threads. The same might not apply to other languages.

My personal preference is for async/await, but it's true that its ecosystem bifurcating virality is a bit of a problem.

ngrilly15• 2 years ago

Could you share an example of a fiber implementation not relying a fat runtime built in the language?

newpavlov9• 2 years ago

https://github.com/Xudong-Huang/may

The project has some serious restrictions and unsound footguns (e.g. around TLS), but otherwise it's usable enough. There are also a number of C/C++ libraries, but I can not comment on those.

anonymoushn7• 2 years ago

For example https://github.com/creationix/libco

The required thing is mostly just to dump your registers on the stack and jump.

zozbot2347• 2 years ago

Because stackful fibers suck for low-level code. See Gor Nishanov's review for the C++ committee http://www.open-std.org/JTC1/SC22/WG21/docs/papers/2018/p136... (linked from https://devblogs.microsoft.com/oldnewthing/20191011-00/?p=10... ). It even sums things up nicely: DO NOT USE FIBERS!

gpderetta16• 2 years ago

To be clear, not everybody agrees with Gor and although they still don't have much traction, stackful coroutines are still being proposed.

Most (but admittedly not all) of Gor issues with stackful coroutines are due to current implementations being purely a library feature with no compiler support. Most of the issues (context switch overhead, tread local issues) can be solved with compiler support.

The issue with lack of support in some existing libraries is unfair, neither do async/await and in fact it would be significantly harder to add support for those as they would require a full rewrite. The take away here is not to try to make M:N fully transparent, not avoid it completely.

The issue with stack usage is real though, split stacks is only a panacea and OS support would be required.

edit: for the other side of the coin, also from microsoft, about issues with the async/await model in Midori (an high performance single-address-space OS): https://joeduffyblog.com/2015/11/19/asynchronous-everything/

They ended up implementing the equivalent of stackful coroutines specifically for performance.

moring• 2 years ago

That paper has some significant flaws:

In section 2.3.1, "Dangers of N:M model", the "danger" lies in using thread-local storage that was built for OS threads, without modification, for stackful fibers. The bottom line here should have been "don't do that", not that stackful fibers are dangerous. Obviously, any library that interacts with the implementation mechanism for concurrency must be built for the mechanism actually used, not a different one.

In section 2.3.2, "Hazards of 1:N model", again points out the dangers of making such blind assumptions, but its main point is that blocking APIs must be blocking on a fiber level, not on the host thread level -- that is, it even proposes the solution for the problem mentioned, then totally ignores that solution. Java's approach (even though, IIRC, N:M) does exactly that: "rewrite it in Java". This is one place where I hope "rewrite it in Rust" becomes more than a meme in the future and actually becomes a foundation for stackful fibers.

Then there are a bunch of case studies that show that you cannot just sprinkle some stackful fibers over an existing codebase and hope for it to work. Surprise: You can't do that with async/await either. "What color is my function" etc.

I'm still hoping for a complete, clearly presented argument for why Java can do it and Rust cannot. (Just for example: "it needs a GC" could be the heart of such an argument).

lamontcg2• 2 years ago

My opinion is that cancellation of tasks fundamentally should be cooperative and uncooperative cancellation is more of a misfeature, which is convenient at the surface level, but has deep issues underneath.

Cooperative cancellation can be pretty annoying with mathematical problems. You can have optimization algorithms calling rootfinding problems calling ODE integrators and any of those can spin for a very long time and you need be threading through cancellation tokens everywhere and the numerical framework generally don't support it. You can and should use iteration counts in all the algorithms but once you're dealing with nested algorithms that can only guarantee that your problem will stop sometime this year, not that it stops within 5 seconds. With these problems I can promise that I'm just doing: lots of math, allocations with their associated page faults, no I/O, writing strings to a standard library Queue object for logging that are handled back in the Main thread that I'm never going to cancel (and whatever other features you think I might need -- which I haven't needed for years now -- I'd be happy to ship that information back to the main thread on a Queue). It feels like that problem should be solvable in the 21st century to me without making me thread cancellation tokens everywhere and defensively code against spinning without checking the token (where I can make mistakes and cause bugs, which I guess you'll just blame me for).

pas• 2 years ago

but why do you want cooperative cancellation? this seems ideal for a thread pool, and if the allocated time is up, ask the OS to stop them. ideally the system would allocate the threads to dedicated CPU cores that have all kinds of interrupt handling disabled (so the throughout is maximalized)

if you want to do this without kernel support, then somehow the program needs to do the periodic checks. the JVM has this (safepoints), but there's no royal road for this.

hsjsbeebue• 2 years ago

What is the language agnostic answer to the same question?

I imagine something to do with memory usage or avoiding thread or thread pool starvation issues. Maybe performance too?

toast04• 2 years ago

I don't have a lot of experience with async/await at high numbers of tasks, but I've run Erlang with millions of processes. It's a lot easier to run millions of Erlang processes on one machine than to run a million OS threads. I suspect async tasks would be similar; an OS thread needs its own stack, and that's going to use at least a page of memory, but often much more. Otoh, an async task or green thread might be able to use less.

If you're running real OS threads, I think task switching is going to be real context switches, which might mean spectre mitigations clear your cpu caches, but task switching can avoid that.

You may end up with more system calls with OS threads, because your runtime might be able to aggregate things a bit (blocking reads become kqueue/epoll/select, but maybe that's actually a wash, because you do still need a read call when the FD is ready, and real blocking only makes a single call)

lmm6• 2 years ago

IMO the biggest reason to avoid threads is simply that it's ~impossible to write safe code using threads (e.g. without race conditions). Arguably with Rust's ownership system that's less true there than in other languages.

kaba02• 2 years ago

Async-await is about concurrency, not parallelism. It can work in both a single-threaded and multi-threaded context, the latter exposing all the typical failure modes of multi-threaded code.

Also, Rust’s ownership model only prevents data races, that’s only the tip of the iceberg of race conditions, and I don’t think that any general model makes it possible to statically determine that any given multithreaded code is safe. Nonetheless, that’s the only way to speed up most kind of code, so possibly the benefits outweigh the cost in many cases.

treflop• 2 years ago

Asynchronous code has race conditions and synchronization issues too.

I pray for all the code written by people who think they didn’t need to learn about synchronization because they wrote asynchronous code.

And unfortunately I’ve come across and had to fix asynchronous code with race conditions.

You cannot escape learning about synchronization. Writing race-condition-free code is not hard.

What is actually hard is writing fast lock-free routines, but that’s more a parallelism problem that affects both threaded and asynchronous code. And most people will never need to reach that level of code optimization for their work.

lelanthran3• 2 years ago

IMO the biggest reason to avoid threads is simply that it's ~impossible to write safe code using threads (e.g. without race conditions).

Javascript has race conditions too, even with no threads involved.

logicchains6• 2 years ago

You can write safe code using threads if you enforce that the only way threads can communicate is by sending messages to each other (via copying, not pointers). This is what Erlang does.

adontz• 2 years ago

There are a lot of moments not covered. For example:

- async/await runs in context of one thread, so there is no need for locks or synchronization. Unless one runs async/await in multiple threads to actually utilize CPU cores, then locks and synchronization are necessary again. This complexity may be hidden in some external code. For example instead of synchronizing access to a single database connection it is much easier to open one database connection per async task. However such approach may affect performance, especially with sqlite and postgres.

- error propagation in async/await is not obvious. Especially when one tries to group up async tasks. Happy eyeballs are a classic example.

- since network I/O was mentioned, backpressure should also be mentioned. CPython implementation of async/await notoriously lacks network backpressure causing some problems.

graphenus• 2 years ago

Async/await just like threads is a concurrency mechanism and also always requires locks when accessing the shared memory. Where does your statement come from?

romanovcode2• 2 years ago

Where does your statement come from?

This is how async/await works in Node (which is single-threaded) so most developers think this is how it works in every technology.

conradludgate• 2 years ago

If you perform single threaded async in Rust, you can drop down to the cheap single threaded RefCell rather than the expensive multithreaded Mutex/RwLock

bsder3• 2 years ago

I have lots of issues with async/await, but this is my primary beef with async/await:

Remember the Gang of Four book "Design Patterns"? It was basically a cookbook on how to work around the deficiencies of (mostly) C++. Yet everybody applied those patterns inside languages that didn't have those deficiencies.

Rust can run multiple threads just fine--it's not Javascript. As such, it didn't have to use async/await. It could have tried any of a bunch of different solutions. Rust is a systems language, after all.

However, async/await was necessary in order to shove Rust down the throats of the Javascript programmers who didn't know anything else. Quoting without.boats:

https://without.boats/blog/why-async-rust/

I drove at async/await with the diligent fervor of the assumption that Rust’s survival depended on this feature.

Whether async/await was even a good fit for Rust technically was of no consequence. Javascript programmers were used to async/await so Rust was going to have async/await so Rust could be jammed down the throats of the Javascript network services programmers--technical consequences be damned.

seabrookmx3• 2 years ago

Threads have a cost. Context switching between them at the kernel level has a cost. There are some workloads that gain performance by multiplexing requests on a thread. Java virtual threads, golang goroutines, and dotnet async/await (which is multi threaded like Rust+tokio) all moved this way for _performance_ reasons not for ergonomic or political ones.

It's also worth pointing out that async/await was not originally a JavaScript thing. It's in many languages now but was first introduced in C#. So by your logic Rust introduced it so it could be "jammed down the throats" of all the dotnet devs..

OtomotO7• 2 years ago

I would damn this, if Async/Await wasn't a good enough (TM) solution for certain problems where Threads are NOT good enough.

Remember: there is a reason why Async/Await was created B E F O R E JavaScript was used for more than sprinkling a few fancy effects on some otherwise static webpages

yxhuvud2• 2 years ago

I generally don't agree with the direction withoutboats went with asynchricity but you are reading in a whole lot more into that sentence than is really there. It is very clear (based on his writing, in this and other articles) that he went with the solution because he thinks it is the right one, on a technical level.

I don't agree, but making it sound like it was about marketing the language to JavaScript people is just wrong.

tsimionescu6• 2 years ago

Async/await was invented for C#, another multithreaded language. It was not designed to work around a lack of true parallelism. It is instead designed to make it easier to interact with async IO without having to resort to manually managed thread pools. It basically codifies at the language level a very common pattern for writing concurrent code.

It is true though that async/await has a significant advantage compared to fibers that is related to single threaded code: it makes it very easy to add good concurrency support on a single thread, especially in languages which support both. In C#, it was particularly useful for executing concurrent operations from the single GUI thread of WPF or WinForms, or from parts of the app which interact with COM. This used the SingleThreadedExecutor, which schedules tasks on the current thread, so it's safe to run GUI updates or COM interactions from a Task, while also using any other async/await code, since tasks inherit their executor.

Karrot_Kream• 2 years ago

async/await is just a different concurrency paradigm with different strengths and weaknesses than threads. Rust has support for threaded concurrency as well though the ecosystem for it is a lot less mature.

guappa3• 2 years ago

Threads are much much slower than async/await.

pkolaczk• 2 years ago

Rust can run multiple threads just fine

Rust is also used in environments which don't support threads. Embedded, bare metal, etc.

tuetuopay2• 2 years ago

Strong disagree.

Rust can run multiple threads just fine--it's not Javascript. As such, it didn't have to use async/await. It could have tried any of a bunch of different solutions. Rust is a systems language, after all.

it allows you to have semantic concurrency where there are no threads available. like, you known, on microncontrollers without an (RT)OS where such a systems programming language is a godsend.

seriously, using async/await on embedded makes so much sense.

withoutboats35• 2 years ago

Every word you've written is false, slanderous and idiotic. You are quoting a post in which I explain at length why async/await was the right fit for Rust technically. You are either illiterate or malignant.

Despite your evident ignorance, there are many network services that are not written in JavaScript. In fact, there are many that are written in C or C++. This is the addressable market of async Rust. Appealing to JavaScript users was not in any way a motivating factor for the development of async/await in Rust. Not at all!

dehrmann• 2 years ago

async can be scarier for locks since a block of code might depend on having exclusive access, and since there wasn't an await, it got it. Once you add an await in the middle, the code breaks. Threading at least makes you codify what actually needs exclusive access.

async also signs you up for managing your own thread scheduling. If you have a lot of IO and short CPU-bound code, this can be OK. If you have (or occasionally have) CPU-bound code, you'll find yourself playing scheduler.

cageface• 2 years ago

Yeah once your app gets to be sufficiently complex you will find yourself needing mutexes after all. Async/await makes the easy parts of concurrency easy but the hard parts are still hard.

junon6• 2 years ago

async/await runs in context of one thread,

Not in Rust.

pohl• 2 years ago

There is a single thread executor crate you can use for that case if it’s what you desire, FWIW.

specialist• 2 years ago

backpressure should also be mentioned

I ran into this when I joined a team using nodejs. Misc services would just ABEND. Coming from Java, I was surprised by this oversight. It was tough explaining my fix to the team. (They had other great skills, which I didn't have.)

error propagation in async/await is not obvious

I'll never use async/await by choice. Solo project, ...maybe. But working with others, using libraries, trying to get everyone on the same page? No way.

I haven't used (language level) structured concurrency in anger yet, but I'm placing my bets on Java's Loom Project. Best as I can tell, it'll moot the debate.

andrewstuart• 2 years ago

> A common refrain is that threads can do everything that async/await can, but simpler.

Who says that? Threads and async/await are different things, and it doesn’t make sense to say one can do what the other does. And threads definitely are not simpler than AA

OtomotO7• 2 years ago

Threads are easier to spawn and that's where all the fuzz comes from, I argue.

Especially in Rust, Async isn't as easy as in other languages with a runtime and it does indeed have some caveats (e.g. cancellation), but all the real fuzz comes from not understanding that they are different strategies for some similar but not the same problems.

It makes little to no sense to use Async/Await for number crunching/CPU-compute intensive tasks, for example.

One can use Threads for some IO waiting though, but it's definitely not the best solution for that particular problem.

To me this whole discussion has two facets:

1) How can Async/Await be more ergonomic in Rust 2) How can we teach people that Async/Await is a different solution with different tradeoffs to Threads - there is a reason why Async/Await was created AFTER WE ALREADY HAD INVENTED THREADS!

dboreham• 2 years ago

Async wasn't invented after threads. It was primarily popularized in a system that has been designed such that it couldn't use threads (the web browser). Everything else is post-justifcation for why it's better. It isn't better.

nurple9• 2 years ago

My problem with rust's async/await is that it's _not_ a different strategy as the continuation tasks _are_ run multithreaded, so it's technically both strategies. IMO one of the biggest selling points of single-threaded async/await was how much complexity falls away compared to managing preemptive synchronization in the multi-threaded case.

I can see why there's so much controversy over async/await in rust. If I had to take both the syntax and cognitive hit of using async/await _and_ multi-threading, I would also angrily call for its removal.

plugin-baby2• 2 years ago

is the difference between threads and async/await more than syntax? or language-specific?

romanovcode2• 2 years ago

Yes, they are different in most cases.

In general await job is to pass the process to 3rd party e.g. database or http and wait for callback whereas thread job is to launch multiple CPU operations in parallel.

jokethrowaway• 2 years ago

Await-async is implemented with a runtime which uses a thread pool or a single thread and allocates work when needed on any thread and waits for IO to yield a result.

With threads you just fully control what blocking code is running on a single thread.

If you are just running computations (or reading files, as filesystem api are not async) it's simpler to just use threads.

plugin-baby2• 2 years ago

or reading files, as filesystem api are not async

This is interesting - all nodejs file system APIs are async by default.

immibis5• 2 years ago

Implementation difference. Threads are usually handled by the kernel and each one has its own stack where you can put anything. And the kernel can switch threads at any time. Async/await has the compiler work out precisely what has to be saved across the marked locations which are the only places context switches can happen. Also it doesn't tell the kernel when it switches context.

gpderetta16• 2 years ago

Threads can be in scheduled in userspace and can also be non-preemptive or have deterministic scheduling.

ikekkdcjkfke• 2 years ago

What i would hope for is implicit async/await so i don't have to write it out / wrap all the time

groestl• 2 years ago

That's eerily similar to what (green) threads are.

hinkley• 2 years ago

I’ve always thought async await is just a less clunky implementation of green threads. Not as a feature parity replacement, but trying to fill the same niche.

yodon• 2 years ago

implicit async/await

I keep suspecting C# will be the place where we see this, but probably not for another couple years.

asabla• 2 years ago

We might not be that far away already. There is this issue[1] on Github, where Microsoft and the community discuss some significant changes.

There is still a lot of questions unanswered, but initial tests look promising.

Ref: https://github.com/dotnet/runtime/issues/94620

anonymoushn7• 2 years ago

We have it in Zig, Lua, Python, C, etc.

t-writescode2• 2 years ago

In Kotlin, only the outer-most wrapping 'suspend' function swap needs to be wrapped; and when you're writing web servers and such, that's handled in the server middleware and so all your code can be `suspend fun blahblah()`

Aardwolf• 2 years ago

Yeah, fully agree with that. To me this is akin to writing line numbers in old BASIC dialects. The language makes the user do something that ought to be automated.

ggm• 2 years ago

As one who struggled with thread safe storage since threads started, the amount of code we carry around with us which looks ok, but turns out not to be viable in threads is remarkably high. I bump into this in C, Python3 quite a lot: you have to work harder to do anything which is not synchronous, no matter how you arrive in that place.

For long lived work, it is not impossible there is no advantage overall to forking a lot of heavyweight processes which operate fast as a single execution state. If you have the cores, the CPU and the memory. Context switching delay is very possibly not your main problem.

For example, I have typically 24 1 hour capture/log files, each 300m+ lines big and I need to ungzip them, parse/grep out some stuff, calculate some aggregates and then recombine. Its a natural pipeline of 3 or 4 processes. The effort to code this into a single language and uplift it to run threads inside, where it's basically 24 forked pipe sequences begs questions: What exactly is going to get faster, when I am probably resource limited by the gunzip process at the front?

You think you can code faster than a grep DFA to select inputs? How sure are you that memory structure is faster than awk? Really sure? I tested some. radix tries are nice and have that log(n) length thing, but AWK was as fast. (IP address "hashing" lookup costs)

(hint: more than one interpreted language simply forks gzip for you, to unzip an input stream)

If you can go to C, then it's possible forking heavyweight processes in C, with good IPC buffer selections or mmap is "good enough"

pjmlp• 2 years ago

As someone that was big into threading back in the 2000, with the experience gathered through times, I think with modern hardware resources we are better off with OS IPC, as it offers much better safety guarantees, specially in C like languages.

coolThingsFirst• 2 years ago

How is async/await implemented under the hood?

lmm6• 2 years ago

Generally via continuations. An async function is transformed to continuation-passing style, await calls with the current continuation, and then you have a runtime that at its simplest is just, like, a queue of tasks and has special-cased primitives for doing things like async I/O where you suspend, and it just pulls tasks off the queue and runs them, and when one task suspends it stores the continuation and runs the next one.

politelemon• 2 years ago

Does that runtime run the tasks across multiple cores?

simonask• 2 years ago

Async/await in Rust is famously not based on continuations, at least not in any traditional sense, where a block of code is passed to a reactor system to be invoked whenever an operation completes.

Instead it is based on "wakers", which are objects associated with a task, and can be used to notify the task's executor that the task is ready to make progress. It is then the job of an executor resume the task. So there is an extra layer of indirection (conceptually).

There are pros and cons, but in essence the system trades a check on resume (often redundant) for the need to make a heap allocation and/or type erasure at each await point.

(It's possible to avoid the latter in continuation-based implementations, like C++ coroutines, but it's pretty hard.)

LtWorf• 2 years ago

poll()

OtomotO7• 2 years ago

"Smart programmers try to avoid complexity. So, they see the extra complexity in async/await and question why it is needed. This question is especially pertinent when considering that a reasonable alternative exists in OS threads."

Hm, yes, but is that complexity really avoided if it's in the language/runtime?

Sure, it's not in your code and it will probably have way more extensive testing and maybe also people that thought about the problem for months and you're just trying to implement Business feature #31514 and not reinventing the wheel, but as someone who has been bitten by specific implementations like the one in Quarkus (not a language, granted), I must say:

Complexity is there to stay (and occasionally bite you), even if it's hidden!

delusional• 2 years ago

I tend to agree, but an important note is that I've never seen an async/await system that didn't ALSO interact with OS threads. Async await is not an alternative to OS threads but rather an additional layer.

OtomotO7• 2 years ago

I have heard about them in embedded systems where there are no OS threads, because there is no OS, but there is Async tasks and a scheduler

IshKebab• 2 years ago

You mean like the async/await systems in the two most popular languages in the world - JavaScript and Python?

Async/await is definitely an alternative to user space threads, especially for IO bound tasks.

grumpyprole• 2 years ago

Hm, yes, but is that complexity really avoided if it's in the language/runtime?

Complexity can be abstracted away and this is still a worthwhile goal. Otherwise, we'd all still be studying Intel CPU documentation.

OtomotO7• 2 years ago

It can and it should, but I disagree with the sentiment that when it's abstracted "away" it's not there anymore.

You just don't see it.

Abstraction doesn't change any fact about how the system actually works.

feverzsj• 2 years ago

Rust async/await is as bad as C++ coroutines. They lack one of the most important language counterparts: async drop/destructor.

OtomotO7• 2 years ago

That's the main gripe imo: Missing features that are not optional for certain problems where the solution would otherwise make perfect sense.

I get the pragmatism to "better ship something that is 80% finished now, than wait for it to be 100% finished in some years", but with Rust's async/await it was released in 2018 and the more time passes, the more it looks like some sharp edges are here to stay.

stevefan1999• 2 years ago

Because you want to avoid context switching cost. In fact, async/await/promise/future/Task is also closely related to fiber as opposed to thread at least in Windows, but fiber is provided from an OS level while async/promise/future/Task is provided by the language itself. You can switch to a different fiber without doing a context switch, just like how you use a state machine to switch to different job using async/promise/future/Task

immibis5• 2 years ago

Context switching cost is mostly artificial. You save kernel/user transitions, but you can also have user-mode threads without transitions. What else are you saving by using coroutines instead of threads? You're not saving register swapping or cache warmup. You are saving stack allocations but those don't cost time.

mrkeen2• 2 years ago

A common refrain is that threads can do everything that async/await can, but simpler.
OS threads don’t require any changes to the programming model, which makes it very easy to express concurrency.

I think these claims need a bit of justification, or else why write the article?

gudzpoz• 2 years ago

I think the author is confusing two things here:

1. User-space threads / Green threads

2. Structured concurrency

The former one is an advantage of async/await, but is not unique to it (see Go or Java Loom for examples that involves no function coloring problem). And the latter one can be implemented with both OS threads and green threads (see Structured concurrency JEPS for Java [1]).

[1] https://openjdk.org/jeps/462

darthrupert2• 2 years ago

My possibly unpopular opinion is that async/await is a mistake as a first class programming construct. That functionality should 100% be in libraries for two reasons: 1) that way it won't infect the actual language in any way and 2) it will be more difficult to use so people will only reach for it if they really need it.

Sync code and threads is the way to go for 99% of the cases where concurrency is needed. Rust handles most of the footguns of that combination anyway.

nurple9• 2 years ago

Couldn't disagree more. In my experience, single-threaded event-loop driven async/await should be used for every possible concurrency need, with the complexity of multi-threaded concurrency being reserved for the rare cases it's needed. As auto-scaled services and FaaS began to become popular, I've found most any need for multithreaded programming almost wholly unnecessary.

zokier2• 2 years ago

The problem is that in Rust the main way of doing async/await (Tokio) is multithreaded by default. So you get all the problems of multithreading, and additionally all the problems with async/await. The discourse would be very different if the default choice of executor would have been thread-local.

t-writescode2• 2 years ago

I'm actually more of a fan of the actor model than the single-threaded event-loop. Individual blobs of guaranteed synchronicity with message passing.

spacechild14• 2 years ago

Not everything is a web backend... CPU bound workloads may be rare in your domain, but I would be careful with generalizations.

darthrupert2• 2 years ago

Perhaps an acceptable solution would be that all blocking things be async by convention, at least potentially. It's the conflict between sync and async that I find distasteful. The coloring problem.

Zig does almost something like this.

Animats4• 2 years ago

Async/await with one thread is simple and well-understood. That's the Javascript model. Threads let you get all those CPUs working on the problem, and Rust helps you manage the locking. Plus, you can have threads at different priorities, which may be necessary if you're compute-bound.

Multi-threaded async/await gets ugly. If you have serious compute-bound sections, the model tends to break down, because you're effectively blocking a thread that you share with others.

Compute-bound multi-threaded does not work as well in Rust as it should. Problems include:

- Futex congestion collapse. This tends to be a problem with some storage allocators. Many threads are hitting the same locks. In particular, growing a buffer can get very expensive in allocators where the recopying takes place with the entire storage allocator locked. I've mentioned before that Wine's library allocator, in a .DLL that's emulating a Microsoft library, is badly prone to this problem. Performance drops by two orders of magnitude with all the CPU time going into spinlocks. Microsoft's own implementation does not have this problem.

- Starvation of unfair mutexes. Both the standard Mutex and crossbeam-channel channels are unfair. If you have multiple threads locking a resource, doing something, unlocking the resource, and repeating that cycle, one thread will win repeatedly and the others will get locked out.[1] If you need fair mutexes, there's "parking-lot". But you don't get the poisoning safety on thread panic that the standard mutexes give you.

If you're not I/O bound, this gets much more complicated.

[1] https://users.rust-lang.org/t/mutex-starvation/89080

exfalso2• 2 years ago

Yes, 100%.

I've mostly only dealt with IO-bound computations, but the contention issues arise there as well. What's the point of having a million coroutines when the IO throughput is bounded again? How will coroutines save me when I immediately exhaust my size 10 DB connection pool? It won't, it just makes debugging and working around the issues harder and difficult to reason about.

AstralStorm3• 2 years ago

The debugging issue is bigger than what it seems.

Use of async/await model in particular ends up with random hanged micro tasks in some random place in code that are very hard to trace back to the cause because they're dispersed potentially anywhere.

Concurrency is also rather undefined, as are priorities most of the time.

This can be partly fixed by labelling, which adds more complexity, but at least is explicit. Then the programmer needs to know what to label... Which they won't do, and Rust has no training wheels to help with concurrency.

Threads, well you have well defined ingress and egress. Priorities are handled by the OS and to some degree fairness is usually ensured.

ninkendo• 2 years ago

What's the point of having a million coroutines when the IO throughput is bounded again?

Because if you did the same thing with a million threads, you’d have all those same problems and more, because (a) threads take up more RAM, (b) threads are more expensive to spawn and tear down, and (c) threads have more registers to save/restore when context switching than coroutines.

The real answer is that you shouldn’t have a million coroutines, and you shouldn’t have a million threads. Doing such a thing is only useful if you have other useful work you can do while you wait on I/O. This is true for web servers (which want to serve other requests, so… maybe one coroutine per active request) and UI’s (which should have a dedicated thread for the UI event loop), but for other applications there’s shockingly little need to actually do async programming in the first place. Parallelism is a well-understood problem and can be used in key places (ie. doing a lot of math that you can easily split into multiple cores), but concurrency is IMO a net negative for most applications that don’t look like a web server.

_flux• 2 years ago

Jemalloc can use separate arenas for different threads which I imagine mostly solves the futex congestion issue. Perhaps it introduces new ones?

gpderetta16• 2 years ago

IIRC glibc default malloc doesn't use per-thread arenas as they would waste too much memory on programs spawning tens of thousands of threads, and glibc can't really make too many workload assumptions. Instead I think it uses a fixed pool of arenas and tries to minimize contention.

These days on Linux, with restartable sequences, you can have true per-cpu arenas with zero contention. Not sure which allocator use them though.

jmspring• 2 years ago

You focus on rust rather than generalizing...

If you are IO bound, consider threads. This is almost the same as async / await.

What was missing above, and the problem with how most compute education is these days, if you are compute bound you need to think about processes.

If you were dealing with python concurrent.futures, you would need to consider processpooexecutor vs. threadpoolexecutor.

Threadpoolexecutor gives you the same as the above.

With multiprocessor executor, you will have multiple processes executing independently but you have to copy a memory space. Which people don't consider. In python DS work - multiprocessor workloads need to determine memory space considerations.

It's kinda f'd up how JS doesn't have engineers think about their workloads.

baq• 2 years ago

Backend JS just spins up another container and/or lambda and if it's too slow and requires multiple CPUs in a single deployment, oh well, too bad.

danbruc3• 2 years ago

[...] if you are compute bound you need to think about processes.

How would that help? Running several processes instead of several threads will not speed anything up [1] and might actually slow you down because of additional inter-process communication overhead.

[1] Unless we are talking about running processes across multiple machines to make use of additional processors.

initplus• 2 years ago

I think you are coming at this from a particular Python mindset, driven by the limitations imposed on Python threading by the GIL. This is a peculiarity specific to Python rather than a general purpose concept about threads vs processes.

Hendrikto• 2 years ago

If you are IO bound, consider threads. This is almost the same as async / await.

Only in Python.

if you are compute bound you need to think about processes.

Also only in Python.

fjdhdhdhfhfhf• 2 years ago

^ kid who only writes python condescending to the engineers actually solving hard problems LOL

neonsunset7• 2 years ago

I keep having to repeat it on this unfortunate website: it's an implementation detail, a multi-threaded executor of async/await can cope with starvation perfectly well as demonstrated by .NET's implementation that can shrug off some really badly written code that interleaves blocking calls with asynchronous.

https://news.ycombinator.com/item?id=39530435

https://news.ycombinator.com/item?id=39786142

https://news.ycombinator.com/item?id=39721626

AstralStorm3• 2 years ago

Said badly with code will still execute with poor performance and the mix can be actively hard to spot.

temporarely4• 2 years ago

Just morning bathroom musings based on your posts (yep /g) and this got me thinking maybe the robust solution (once and for all for all languages) may require a rethink at the hardware level. The CPU bound issue comes down to systemic interrupt/resume I think; if this can be done fairly for n wip thread-of-execution with efficient queued context swaps (say maybe a cpu with n wip contexts) then the problem becomes a resource allocation issue. Your thoughts?

binary1322• 2 years ago

What you said sounded in my head more like you’re describing a cooperatively scheduled OS rather than a novel hardware architecture.

Animats4• 2 years ago

a cpu with N wip contexts

That's what "hyper-threading" is. There's enough duplicated hardware that beyond 2 hyper-threads, it seems to be more effective to add another CPU. If anybody ever built a 4-hyperthread CPU, it didn't become a major product.

It's been tried a few times in the past, back when CPUs were slow relative to memory. There was a National Semiconductor microprocessor where the state of the CPU was stored in main memory, and, by changing one register, control switched to another thread. Going way back, the CDC 6600, which was said to have 10 peripheral processors for I/O, really had only one, with ten copies of the state hardware.

Today, memory is more of a bottleneck that the CPU, so this is not a win.

klysm• 2 years ago

I find this has worked well for me when I can easily state what thread pool work gets executed on.

mmis1000• 2 years ago

I think its kotlin's model. The language don't have a default co-routine executor set, you need to provide your own or spawn one with the std. It can be thread-pooled, single thread, or custom one but there isn't a default one set.

If you use single threaded executor, then race condition won't happen. If you choose pooled, then you obviously should realize there can be a race condition. It's all about choice you made.

HarHarVeryFunny• 2 years ago

I'm not sure what you mean by "multi-threaded async/wait"... Isn't the article considering async/await as an alternative to threads (i.e. coroutines vs threads)?

I'm a C++ programmer, and still using C++17 at work, so no coroutines, but don't futures provide a similar API? Useful for writing async code in serialized fashion that may be easier (vs threads) to think about and debug.

Of course there are still all the potential pitfalls that you enumerate, so it's no magic bullet for sure, but still a useful style of programming on occasion.

lights0123• 2 years ago

async/await tasks can be run in parallel on multiple threads, usually no more threads than there are hardware threads. This allows using the full capabilities of the machine, not just one core's worth. In a server environment with languages that support async/await but don't have the ability to execute on multiple cores like Node.js and Python, this is usually done by spawning many duplicate processes and distributing incoming connections round-robin between them.

zarzavat• 2 years ago

They mean async/await running over multiple OS threads compared to over one OS thread.

You can also have threads running on one OS thread (Python) or running on multiple OS threads (everything else).

Every language’s concurrency model is determined by both a concurrency interface (callbacks, promises, async await, threads, etc), and an implementation (single-threaded, multiple OS threads, multiple OS processes).

miohtama• 2 years ago

Also there is the extra software development and maintenance cost due to coloured functions that async/await causes

https://journal.stuffwithstuff.com/2015/02/01/what-color-is-...

Unless you are doing high scalability software, async might not be worth of the trade offs.

nine_k2• 2 years ago

If you expect to color your functions async by default, it's really easy to turn a sync functuion into a near-zero-cost async, a Future that has already been resolved at construction time by calling the sync function.

This way, JS / TS becomes pretty comfortable. Except for stacktraces, of course.

MuffinFlavored3• 2 years ago

Async/await with one thread is simple and well-understood. That's the Javascript model.

Bit of nuance (I'm not an authority of this and I don't know the current up-to-date answer across 'all' of the Javascript runtimes these days):

Isn't it technically "the developer is exposed the concept of just one thread without having to worry about order of execution (other than callbacks/that sort of pre-emption)" but under the hood it can actually be using as many threads as it wants (and often does)? It's just "abstracted" away from the user.

recursive2• 2 years ago

I don't think so. Two threads updating the same variable would be observable in code. There's no way around it.

kaba02• 2 years ago

I mean, yeah, if you go deep enough your OS may decide to schedule your browser thread to a different core as well. I don’t think it has any relevance here - semantically, it is executed on a single thread, which is very different from multi-threading.

sporkland• 2 years ago

I don't know how the challenges with cooperative (no-preemptive) multitasking keep needing to get rediscovered. Even golang, which I consider a very responsibly designed language, went with cooperative at first until they were forced to switch to preemptive. Not saying cooperative multitasking doesn't have its place, just that it's gotta have a warning sticker or even better disallow certain types of code from executing statically.

Also great time to plug a related post, What color is your function:

https://journal.stuffwithstuff.com/2015/02/01/what-color-is-...

exfalso2• 2 years ago

It's interesting to see an almost marketing-like campaign to save face for async/await. It is very clear from my experience that it was not only a technical mistake, it also cost the community dearly. Instead of focusing on language features that are actually useful, the Rust effort has been sidetracked by this mess. I'm still very hopeful for the language though, and it is the best thing we've got at the moment. I'm just worried that this whole fight will drag on forever. P.S the AsyncWrite/AsyncRead example looks reasonable, but in fact you can do the same thing with threads/fds as long as you restrict yourself to *nix.

logicchains6• 2 years ago

It's not a technical mistake, it's a brilliant solution for when you need ultra-low-latency async code. The mistake is pushing it for the vast majority of use-cases where this isn't needed.

surajrmal2• 2 years ago

The reason it's pushed everywhere is because it only works well if the ecosystem uses it. If the larger rust ecosystem used sync code in most places, async await in rust would be unusable by large swaths of the community.

speed_spread• 2 years ago

It's not pushed, it's pulled by the hype. The Rust community got onboard that train and started to async all the things without regards to actual need or consequences.

ykonstant• 2 years ago

Instead of focusing on language features that are actually useful, the Rust effort has been sidetracked by this mess.

I don't know if you are correct or not (I am not very familiar with Rust) but empirically 9/10 Rust discussions I see nowadays on HN/reddit do revolve around async. It kinda sucks for me because I don't care about async at all, and I am interested in reading stuff about Rust.

klabb32• 2 years ago

empirically 9/10 Rust discussions I see nowadays on HN/reddit do revolve around async. It kinda sucks for me because I don't care about async at all, and I am interested in reading stuff about Rust.

100% yes. I really feel bad precisely for people in your situation. But, there’s a good reason why you see this async spam on HN:

(I am not very familiar with Rust)

Once you get past the initial basics (which are amazing, with pattern matching, sum types, traits, RAII, even borrowing can be quite fun), you’ll want to do something “real world”, which typically involves some form of networked IO. That’s when the nightmare begins.

So there’s traditional threaded IO (mentioned in the post), which lets you defer the problem a bit (maybe you can stay on the main thread if eg you’re building a CLI). But every crate that does IO needs to pick a side (async or sync). And so the lib you want to use may require it, which means you need to use async too. There are two ways of doing the same thing - which are API incompatible - meaning if you start with sync and need async later - get ready for a refactor.

Now, you (and the crates you’re using) also have to pick a faction within async, ie which executor to use. I’ve been OOTL a while but I think it’s mostly settled on Tokio these days(?), which probably is for the best. There are even sub-choices about Send-ness (for single-vs-multithreaded executors) and such that also impact API-compatibility.

In either case, a helluva lot of people with simple use-cases absolutely need to worry about async. This is problematic for three main reasons: (1) these choices are front-loaded and hard to reverse and (2) async is more complex to use, debug and understand, and (3) in practice constrains the use of Rust best-practice features like static borrowing.

I don’t think anyone questions the strive for insane performance that asynchronous IO (io_uring, epoll etc) can unlock together with low-allocation runtimes. However, async was supposed to be an ergonomic way to deliver that, ideally without splitting the ecosystem into camps. Otherwise, perhaps just do manual event looping with custom state machines, which doesn’t need any special language features.

Ygg23• 2 years ago

It's true for general programming focused Reddits, but eh, those will always try to poke fun at languages from the extreme perspective. E.g. they are a problem for people new to a language but not for others that have gone through learning the language extensively.

Remember, there are only two types of languages - languages no one uses and languages people complain about. When was the last time you heard Brainfuck doesn't give you good tools to access files?

On /r/rust Reddit, most talk is about slow compile/build times.

asa4005• 2 years ago

Honestly: just ignore them. Just start using Rust! It's a lovely, useful language. HN/reddit are not representative most of the people out there in the real world writing Rust code that solves problems. I am not saying their concerns are invalid, but there is a tendency on these forums to form a self-reinforcing collective opinion that is overfit to the type of people who like to spend time on these forums. Reality is almost always richer and more complicated.

junon6• 2 years ago

I've used async in firmware before. It was a lifesaver. The generalizations you make are unfounded and are clearly biased toward a certain workload.

AstralStorm3• 2 years ago

I'd love the detail on this, what did it save you from and how did you ensure your firmware does not, say, hang?

jerf4• 2 years ago

I personally agree that it is great that Rust as a language is able to function in an embedded environment. Someone needs to grasp that nettle. I started by writing "concede" in my first sentence there but it's not a concession. It's great, I celebrate it, and Rust is a fairly good choice for at least programmers working in that space. (Whether it's good for electrical engineers is another question, but that's a debate for another day.)

However, the entire Rust library ecosystem shouldn't bend itself around what is ultimately still a niche use case. Embedded uses are still a small fraction of Rust programs and that is unlikely to change anytime soon. I am confident the vast bulk of Rust programs and programmers, even including some people vigorously defending async/await, would find they are actually happier and more productive with threads, and that they would be completely incapable of finding any real, perceptible performance difference. Such exceptions as there may be, which I not only "don't deny" exist but insist exist, are welcome to pay the price of async/await and I celebrate that they have that choice.

But as it stands now, async/await may be the biggest current premature optimization in common use. Though "I am writing a web site that will experience upwards of several dozen hits per minute, should I use the web framework that benches at 1,000,000 reqs/sec or 2,000,000 reqs/sec?" is stiff competition.

guappa3• 2 years ago

If you think that threads are faster than poll(), I would like to know in what use case that happens, because I have never once encountered this in my life.

Ygg23• 2 years ago

It's interesting to see an almost marketing-like campaign to save face for async/await.

It's not, it's just Rust people want to explain choices that led them here, and HN/Reddit crowd is all about async controversy, so more Rust people make blog about it, so more HN/Reddit focus on it. Like any good controversy, it's a self-reinforcing cycle.

it also cost the community dearly

Citation needed. Having async/await opened a door to new contributors as well, it was a REALLY requested feature, and it made stuff like Embassy possible.

And it made stuff like effects and/or keyword generics a more requested feature.

uobytx2• 2 years ago

Do you have any evidence to back up the claim that the async efforts have taken away from other useful async features?

Also, lots of major rust projects depend on async for their design characteristics, not just for the significant performance improvements over the thread-based alternatives. These benefits are easy to see in just about any major IO-bound workload. I think the widespread adoption of async in major crates (by smart people solving real world problems) is a strong indicator that async is a language feature that is "actually useful".

The fight is mostly on hackernews and reddit, and mostly in the form of people who don't need async being upset it exists, because all the crates they use for IO want async now. I understand that it isn't fun when that happens, and there are clearly some real problems with async that they are still solving. It isn't perfect. But it feels like the split over async that is apparent in forum discussions just isn't nearly as wide or dramatic in actual projects.

asa4005• 2 years ago

I have to respectfully disagree. People are allowed to like stuff, it doesn't make it a marketing campaign or a conspiracy. This negativity on async, as if it's just a settled debate that async is a failure and therefor Rust is a failure, feels self-reinforcing.

I've written a fair amount of Rust both with threads and with async and you know what? Async is useful, very often for exactly the reasons the OP mentions, not necessarily for performance. I don't like async for theoretical reasons. I like it because it works. We use async extensively at my job for a lot of lower-level messaging and machine-control code and it works really well. Having well-defined interfaces for Future and Stream that you can pass around is nice. Having a robust mechanism for timers is nice. Cancellation is nice. Tokio is actually really solid (I can't speak to the other executors). I see this "async sucks" meme all over the various programming forums and it feels like a bunch of people going "this thing, that isn't perfect, is therefor trash". Are we not better than this?

This is not to say that async doesn't have issues. Of course it does. I'm not going to enumerate them here, but they exist and we should work to fix them. There are plenty of legitimate criticisms that have been made and can continue to be made about Rust's concurrency story, but async hasn't "cost the community dearly". People who wouldn't use Rust otherwise are using async every single day to solve real problems for real users with concurrent code, which is quite a bit more than can be said for all kinds of other theoretical different implementations Rust could have gone with, but didn't.

jayd163• 2 years ago

Another discussion where people don't get async/await, can't fathom why you would want a concurrency mechanism on a single thread and assume no one needs it.

UI programming, communication with the GPU, and cross runtime communication are good examples but I'm sure there are more.

Threads, green or otherwise, don't work for those cases but async/await does.

mike_hearn14• 2 years ago

You can easily use threads with GUIs and I've written a bunch of GUI apps in the past that exploited threads quite effectively.

jayd163• 2 years ago

What popular GUI framework is multi threaded?

anonymoushn7• 2 years ago

It's definitely important to be able to manage a bunch of tasks explicitly in a single thread. If there are any other possible implementations of language features that result in the same binary and cause less trouble for the language's users, let's chat about those :)

anacrolix• 2 years ago

async/await is syntax. Most of what people associate with it is actually the benefit of virtual threading. Asynchrony is how virtual threading is achieved in user space. Rust has direct native threading by default and can't do virtual threading without a runtime. Async/await makes the use of a runtime explicit. It could be possible to have both virtual and OS threading without special syntax but it would require a marker trait, algebraic effects or monads. Without those things you either need to choose user or OS level threads by default. If user level threads are the default, you need a runtime by default. (Even if that runtime is only for concurrency.). Go made that the default. Again, you don't need async/await to have virtual threading.

tsimionescu6• 2 years ago

Async/await is syntax, but it's not equivalent to virtual threads. It is special syntax for a certain pattern of writing concurrent (and possibly parallel) code: one in which you launch a concurrent operation specifically to get back one result. When you start an OS thread or a virtual thread, that thread can do anything. When you launch a task, it can only do one thing: return one result of the type you asked for.

Async/await is perfect for operations that map well onto this structure. For example, most IO reads and writes fit well into this model - you trigger the IO operation to get back a specific result, then do some other work while it's being prepared, and when you need the result, you block until it's available. Other common concurrent operations don't map well on this at all - for example, if you want to monitor a resource to do something every time its state changes, then this is not well modeled at all as a single operation with a unique result, and a virtual or real thread would be a much better option.

Also, having both virtual and OS threads doesn't need any special syntax. You just need two different functions for creating a thread - StartOSThread(func) and StartVirtualThread(func), and a similar split for other functions that directly interact with threads (join, cancel, etc), and a function for telling whether you are currently running in a virtual thread. Everything else stays the same. This is what Java is doing with Project Loom, I'm not speaking just in principle.

The huge difficulty with virtual threads is implementing all blocking operations (IO, synchronization primitives, waits, etc) such that they use async OS primitives and yield to the virtual thread scheduler, instead of actually blocking the OS thread running the virtual thread.

mike_hearn14• 2 years ago

You don't technically need two ways to start threads. That's how Java does it, and there's some technical reason for it that I always forget. There are edge cases where virtual and physical threads aren't completely interchangeable.

gpderetta16• 2 years ago

Aside from the fact that an async/await operation can have arbitrary side effects , returning values is not a prerogative of async/await. For example in c++:

  void foo() {
     auto future = std::async([]{ /* do some stuff */; return some_value; });
     ... // do some work concurrently

     // join the async operation
     auto some_value = future.get();
  }

Here std::async starts a thread (or potentially queues a task in a thread pool); please don't confuse async/await (i.e. stackless coroutines), with more general future/promise and structured concurrency patterns.

edit: note I'm not making any claims that std::async, std::future, std::promise are well designed.

anonymousDan• 2 years ago

I feel like the last difficulty should potentially be easier with the emergence of IO uring support for a wider variety of system calls. Would you agree?

littlestymaar24• 2 years ago

But at the same time, the syntax itself is what matters to most of its users. OS thread can be fast enough for most workloads, but having the syntax showing up where the IO is happening, easily run concurrently what can be, and cancel stuff after a timeout, is the superpower of async/await.

Discussing about implantation performance is missing the forest for the tree. It's like comparing exceptions to error enums on that prism alone: sure there are some implementation where this or that is more efficient than the other, but the ability to express intent in the code is what make those killer features.

globular-toast• 2 years ago

I recall that the thing that made me stick with Rust is the Iterator trait. It blew my mind that you could make something an Iterator, apply a handful of different combinators, then pass the resulting Iterator into any function that took an Iterator.

What languages did the author use before? Is this any different from the interface pattern seen in many other languages?

KaiserPro• 2 years ago

I think its partly down to what you grew up with. If you're of the JS (or python 3) generation then you're _probably_ more comfortable with async.

However there are things where async seems to be more of a semantic fit for what youre doing.

for example FastAPI is all async, and it makes sense. I started using it, because it scaled better than anything else. They have don't a nice job of making the interface as painless as possible. It almost doesn't feel like surprise goto.

I do a lot of stream processing, so for me threads is a better fit. It scales well enough, and should I need to either go to multiprocessing (not great) or duplicate to a new stand-alone process, its fairly simple (Keeping everything message based also helps.)

async/threads is _almost_ like shop bought coke vs sodastream. They are mostly the same, but have slightly different semantics.

avodonosov7• 2 years ago

Issues with the article:

1. Only one example is given (web server), solved incorrectly for threads. I will elaborate below.

2. The question is framed as if people specifically want OS threads instead of async/await .

But programmers want threads conceptually, semantically. Write sequential logic and don't use strange annotations like "async". In other words, if async/await is so good, why not make all functions in the language implicitly async, and instead of "await" just write normal function calls? Then you will suddenly be programming in threads.

OS threads are expensive due to statically allocated stack, and we don't want that. We want cheap threads, that can be run in millions on a single CPU. But without the clumsy "async/await" words. (The `wait` word remains in it's classic sense: when you wait for an event, for another thread to complete, etc - a blocking operation of waiting. But we don't want it for function invocations).

Back to #1 - the web server example.

When timeout is implemented in async/await variant of the solution, using the `driver.race(timeout).await`, what happens to the client socket after the `race` signals the timeout error? Does socket remain open, remains connected to the client - essentially leaked?

The timeout solution for threaded version may look almost the same, as it looks for async/await: `threaded_race(client_thread, timeout).wait`. This threaded_race function uses a timer to track a timeout in parallel with the thread, and when the timeout is reached it calls `client_thread.interrupt()` - the Java way. (The `Thread.interrupt()`, if thread is not blocked, simply sets a flag; and if the thread is blocked in an IO call, this call throws an InterruptedException. That's a checked exception, so compiler forces programmer to wrap the `client.read_to_end(&mut data)` into try / catch or declare the exception in the `handle_client`. So programmer will not forget to close the client socket).

littlestymaar24• 2 years ago

But programmers want threads conceptually, semantically. Write sequential logic and don't use strange annotations like "async". In other words, if async/await is so good, why not make all functions in the language implicitly async, and in stead of "await" just use normal function calls? Then you will suddenly be programming in threads.

Some programmers do, but many want exactly the opposite as well. Most of the time I don't care if it's an OS blocking syscall or a non-blocking one, but I do care about understanding the control flow of the program I'm reading and see where there's waiting time and how to make them run concurrently.

In fact, I'd kill to have a blocking/block keyword pair whenever I'm working with blocking functions, because they can surreptitiously slow down everything without you paying attention (I can't count how many pieces of software I've seen with blocking syscalls in the UI thread, leading to frustratingly slow apps!).

avodonosov7• 2 years ago

But all functions are blocking.

   fn foo() {bar(1, 2);}
   fn bar(a, b) {return a + b;}

Here bar is a blocking function.

mike_hearn14• 2 years ago

This is a really common comment to see on HN threads about async/await vs fibers/virtual threads.

What you're asking for is performance to be represented statically in the type system. "Blocking" is not a useful concept for this. As avodonosov is pointing out, nothing stops a syscall being incredibly fast and for a regular function that doesn't talk to the kernel at all being incredibly slow. The former won't matter for UI responsiveness, the latter will.

This isn't a theoretical concern. Historically a slow class of functions involved reading/writing to the file system, but in some cases now you'll find that doing so is basically free and you'll struggle to keep the storage device saturated without a lot of work on multi-threading. Fast NVMe SSDs like found in enterprise storage products or MacBooks are a good example of this.

There are no languages that reify performance in the type system, partly because it would mean that optimizing a function might break the callers, which doesn't make sense, and partly because the performance of a function can vary wildly depending on the parameters it's given.

Async/await is therefore basically a giant hack and confuses people about performance. It also makes maintenance difficult. If you start with a function that isn't async and suddenly realize that you need to read something from disk during its execution, even if it only happens sometimes or when the user passes in a specific option, you are forced by the runtime to mark the function async which is then viral throughout the codebase, making it a breaking change - and for what? The performance impact of reading the file could easily be lost in the noise on modern hardware and operating systems.

The right way to handle this is the Java approach (by pron, who is posting in this thread). You give the developer threads and make it cheap to have lots of them. Now break down tasks into these cheap threads and let the runtime/OS figure out if it's profitable to release the thread stack or not. They're the best placed to do it because it's a totally dynamic decision that can vary on a case-by-case basis.

fragmede• 2 years ago

There's also writing your code with poll() and select(), which is its own thing.

tuetuopay2• 2 years ago

well that's the great thing with async rust: you write with poll and select without writing poll and select. let the computer and the compiler get this detail out of my way (seriously I don't want to do the fd interest list myself).

and I can still write conceptually similar select code using the, well, select! macro provided by most async runtimes to do the same on a select list of futures. better separation, easier to read, and overall it boils down to the same thing.

f_devd5• 2 years ago

When timeout is implemented in async/await variant of the solution, using the `driver.race(timeout).await`, what happens to the client socket after the `race` signals the timeout error?

Any internal race() values will be `Drop`ed and driver itself will remain (although rust will complain you are not handling the Result if you type it 'as is'), if a new socket was created local to the future it will be cleaned up.

The niceness of futures (in Rust) is that all the behavior around it can be defined, while "all functions are blocking." as you state in a sibling comment, Rust allows you to specify when to defer execution to the next task in the task queue, meaning it will poll tasks arbitrarily quickly with an explicitly held state (the Future struct). This makes it both very fast (compared to threads which need to sleep() in order to defer) and easy to reason about.

Java's Thread.interrupt is also just a sleep loop, which is fine for most applications to be fair. Rust is a system language, you can't have that in embedded systems, and it's not desirable for kernels or low-latency applications.

avodonosov7• 2 years ago

Java's Thread.interrupt is also just a sleep loop

You probably mean that Java's socket reading under the hood may start a non-blocking IO operation on the socket, and then run a loop, which can react on Thread.interrupt() (which, in turn, will basically be setting a flag).

But that's an implementation detail, and it does not need to be implemented that way.

It can be implemented the same way as async/await. When a thread calls socket reading, the runtime system will take the current threads continuation off the execution, and use CPU to execute the next task in the queue. (That's how Java's new virtual threads are implemented).

Threads and async/await are basically the same thing.

So why not drop this special word `async`?

imtringued2• 2 years ago

In other words, if async/await is so good, why not make all functions in the language implicitly async, and instead of "await" just write normal function calls? Then you will suddenly be programming in threads.

Why not go one step further and invent "Parallel Rust"? And by parallel I mean it. Just a nice little keyword "parallel {}" where every statement inside the parallel block is executed in parallel, the same way it is done in HDLs. Rust's borrow checker should be able to ensure parallel code is safe. Of course one problem with this strategy is that we don't exactly have processors that are designed to spawn and process micro-threads. You would need to go back all the way to Sun's SPARC architecture for that and then extend it with the concept of a tree based stack so that multiple threads can share the same stack.

avodonosov7• 2 years ago

That would be a good step forward, I support it :)

BTW, do we need the `parallel` keyword, or better to simply let all code be parallel by default?

zozbot2347• 2 years ago

Just a nice little keyword "parallel {}" where every statement inside the parallel block is executed in parallel, the same way it is done in HDLs. Rust's borrow checker should be able to ensure parallel code is safe.

The rayon crate lets you do something quite similar.

gpderetta16• 2 years ago

You do not need to spawn threads/tasks eagerly. You can do it lazily on work-stealing. See cilk++.

galangalalgol3• 2 years ago

Doesn't rayon have a syntax like that?

jerf4• 2 years ago

I believe the answer is "that implies a runtime", and Rust as a whole is not willing to pull that up into a language requirement.

This is in contrast to Haskell, Go, dynamic scripting languages, and, frankly, nearly every other language on the market. Almost everything has a runtime nowadays, and while each individually may be fine they don't always play well together. It is important that as C rides into the sunset (optimistic and aspirational, sure, but I hope and believe also true) and C++ becomes an ever more complex choice to make for various reasons that we have a high-power very systems-oriented programming language that will make that choice, because someone needs to.

valenterry5• 2 years ago

In other words, if async/await is so good, why not make all functions in the language implicitly async, and instead of "await" just write normal function calls? Then you will suddenly be programming in threads.

It has been tried various times in the last decades. You want to search for "RPC". All attempts at trying to unify sync and async have failed, because there is a big semantical difference between running code within a thread or between threads or even between computers. Trying to abstract over that will eventually be insufficient. So better learn how to do it properly from the beginning.

valenterry5• 2 years ago

Oh and just to add onto it, I think async/await is not really the best solution to tackle these semantic difference. I prefere the green-thread-IO approach, which feels a might more heavy but it leads to a true understanding how to combine and control logic in a concurrent/parallel setting. Async/await is great to add it to languages that already have something like promises and want to improve syntax in an easy way though, so it has its place - but I think it was not the best choice for Rust.

toast04• 2 years ago

I think you've got some of this in your own reply, but ... I feel like Erlang has gone all in on if async is good, why not make everything async. "Everything" in Erlang is built on top of async messaging passing, or the appearance thereof. Erlang hasn't taken over the world, but I think it's still successful; chat services descended from ejabberd have taken over the world; RabbitMQ seems pretty popular, too. OTOH, the system as a whole only works because Erlang can be effectively preemptive in green threads because of the nature of the language. Another thing to note is that you can build the feeling of synchronous calling by sending a request and immediately waiting for a response, but it's vary hard to go the other way. If you build your RPC system on the basis of synchronous calls, it's going to be painful --- sometimes you want to start many calls and then wait for the responses together, that gets real messy if you have to spawn threads/tasks every time.

Kinrany4• 2 years ago

In other words, if async/await is so good, why not make all functions in the language implicitly async, and instead of "await" just write normal function calls?

IIRC withoutboats said in one of the posts that the true answer is compatibility with C.

marcosdumay2• 2 years ago

if async/await is so good, why not make all functions in the language implicitly async, and instead of "await" just write normal function calls?

You mean like Haskell?

The answer is that you need an incredibly good compiler to make this behave adequately, and even then, every once in a while you'll get the wrong behavior and need to rewrite your code in a weird way.

cryptonector10• 2 years ago

OS threads are expensive due to statically allocated stack, and we don't want that. We want cheap threads, that can be run in millions on a single CPU. But without the clumsy "async/await" words.

Green threads ("cheap threads") are still expensive if you end up spreading a lot of per-client state on the stack. That's because with async/await and CPS you end up compressing the per-client state into a per-client data structure, and you end up having very few function call activation frames on the stack, all of which unwind before blocking in the executor.

nottorp• 2 years ago

No offense but you don't serve 1 million clients with 1 thread per client.

So what this ends up as is describing the mechanism that Rust uses to hide the state machine you'd do by hand in an older language.

ardies• 2 years ago

Just tried changing my python code to use AA instead of the 2 threads I use and it just complicated the code more. I had to create a thread anyway to deal with the tkinter event loop issue. And then I still have to check inside the functions if the task is still active. But the worst part is converting all the stack of functions to to be async/await.

bsaul• 2 years ago

"Obviously, the embryonic web tried to solve this problem. The original solution was to introduce threading"

Are we calling 1970's mainframes "the embryonic web" now ?

immibis5• 2 years ago

Well, apparently they did consist of a lot of tangled threads...

DaiPlusPlus• 2 years ago

They’re beautiful: https://www.embecosm.com/app/uploads/Cray_1_Wiring.jpg

mike_hearn14• 2 years ago

That sentence is super confusing. The web has never supported multi-threading.

paulsutter• 2 years ago

If you are doing anything complex you really want to use a polling loop. That’s how video games, avionics systems, industrial equipment, etc is programmed

guappa3• 2 years ago

That's what async/await abstracts internally.

samsquire• 2 years ago

Thank you for this post, very interesting.

Parallelism and async and multithreading is my hobby.

I think nodejs and browsers made typeless but coloured async easily digested and understood by everyone. Promises for example.

The libuv event loop used by nodejs is powerful for single threaded IO scalability, but I am looking for something that is multicore and scalable.

I like thread-per-core architectures.

The code written to an architecture has a high overhead in terms of mental understandability to new colleagues and barrier to transform it. Refactoring Postgres multiprocess implementation or nginx's worker model, it would be a major effort and undertaking. Firefox did a lot of work with Electrolysis Quantum to make it multiprocess.

I was yesterday and recently trying to implement a lockfree work queue that any thread can pick up work and "steal" work from other threads.

I have a multithreaded runtime I am building in C which is a "phaser", it does synchronization in bulk. So rather than mutexes, you have mailboxes which are queues.

neonsunset7• 2 years ago

Please. Stop posting about async/await here. It just gives space for everyone and their mother to post their superstitions and misconceptions about what is otherwise an elegant and extremely powerful model.

The Rust implementation is great, so is the C# one. They have their own tradeoffs but I would never choose anything else, and 8 out of 10 developers who do disagree never stopped reading past "me sees await means no thread blocky" instead of focusing on structured concurrency patterns it enables. Hell, C# does not even need any fancy terms for this because it is this low ceremony. Worse languages, however, do require more effort so have to justify it with inventing words.

sapiogram• 2 years ago

Please. Stop posting about async/await here.

I agree with everything else you said, but that's all the more reason to post it imo. The misconceptions aren't going away just because you don't hear about them, and every time I hear the different arguments, my own understanding grows a little bit. It's annoying, but quite healthy.

EVa5I7bHFq9mnYK2• 2 years ago

Threads are not useful for I/O bound code. Anyway the requests are getting serialized into a single queue within the network driver or a disk driver. Actually, they make things worse by 1) issuing multiple out-of-order request to the disk, and 2) wasting time and memory switching that thread context back and forth. For CPU-bound tasks, it makes sense to create multiple threads, but only up to amount of actual cores available, otherwise it makes things worse again by cache thrashing. Thread pool(s) + message queues is what worked best for me.

gpderetta16• 2 years ago

Any semi-modern card or ssd provides multiqueue support in hardware.

And even with a single queue, concurrency can help keeping a queue full.

simpaticoder• 2 years ago

Async/await vs threads is yet another entry in the ~800 volume ongoing series "Where the Indirection Go?" In this case, you put concurrency inside the process (async) or outside the process (threads). Your CPU is, in both cases, constantly rotating between workloads, and that indirection can be inside or outside the process. Once you start caring about either human usability or fractional performance, the distinction matters. Otherwise, it doesn't.

makerdiety• 2 years ago

Because asynchronous programming reduces to its equivalent in digital circuit design being contingent on analog circuit engineering decisions.

runiq• 2 years ago

Sorry, not a native speaker here and I can't parse that sentence. Would you be so kind to dumb it down a little?

zaphar4• 2 years ago

The complaints around async/await vs threads to my mind have not been that one is more or less complex than the other. It is that it bifurcates the ecosystem and one of them ends up being a second class citizen causing friction when you choose the wrong one for your project.

While you can mix and match them it's hacky and inefficient when you need to. As it stands now the Rust ecosystem has decided that if you want to do anything involving IO you are stuck with an all async/await ecosystem. Since nearly everything you might want to do in Rust probably involves IO with very few exceptions that means for the most part you should probably ignore non-async libraries regardless of whether the rest of your application wants it to be async or not.

There is a hypothetical world where Rust used abstractions that are even more composable than async/await, whose composability really wants everything else to be async/await too. If that had happened then I think most of the complaints would have disappeared.

HideousKojima• 2 years ago

As it stands now the Rust ecosystem has decided that if you want to do anything involving IO you are stuck with an all async/await ecosystem.

I mean C# works basically the same way, even through there are non-async options for IO using the async options basically forces you to be async all the way back to Main(). There are ways to safely call async methods from sync methods but they make debugging infinitely harder.

zaphar4• 2 years ago

Well, yes. That doesn't mean it's not annoying though. It happens in every language that provides syntactic support for the distinction between async/await and non async. It's, I think, core to the syntactic and semantic abstractions that were popularized by Javascript.

K0nserv3• 2 years ago

I agree with your diagnosis. It's what I concluded in my own Rust async blog post[0](which surely are mandatory now). It's even worse than bifurcating the ecosystem because even within async code it's almost always closely tied to the executor, usually Tokio. I talk about this as an extension to function colouring, adopting without.boats's three colour proposition with blue(non-IO), green(blocking-IO), and red(async-IO). In the extended model it's really blue, green, red(Tokio), purple(async-std), and orange(smol) etc.

I find that the sans-IO pattern is the best solution to this problem. Under this pattern you isolate all blue code and use inversion of control for I/O and time. This way you end up with the core protocol logic being unaware of IO and it becomes simple to wrap it in various forms of IO.

0: https://hugotunius.se/2024/03/08/on-async-rust.html

fsociety• 2 years ago

This is a cool pattern, thanks for the share.

KiloCNC• 2 years ago

I love the fact that people outside the Python ecosystem are spreading the word about sans-IO. I think it should be the next iteration in coding using futures-based concurrency. I only wish it were more popular in the Python land as well.

Dessesaf• 2 years ago

Is that sans-IO pattern doing anything particularly novel, or is that basically just a subset of what any functional programmer does anyway?

Don't get me wrong, I'm fully on board with isolating IO. But why not go the slight extra step and just make it completely pure? You've already done the hard part of purity. The rest is easy.

Then you get all those nice benefits of being generic over async and sync, but can also memoize and parallelize freely, and all the other benefits of purity.

Kinrany4• 2 years ago

Since nearly everything you might want to do in Rust probably involves IO with very few exceptions that means for the most part you should probably ignore non-async libraries regardless of whether the rest of your application wants it to be async or not.

Only if you have two libraries to choose from and they are otherwise identical, which is rare. Using blocking code in async applications is not as seamless as it should be but not hard. Instead of writing `foo()` you write `tokio::spawn_blocking(foo).await`. It will run the new code in a separate thread and return a future that will resolve once that thread is done.

Kinrany4• 2 years ago

And of course most libraries don't even need IO because the application can do it for them, so it only makes sense for them to be async if they're computationally heavy enough to cause problems for the runtime.

zaphar4• 2 years ago

That assumes you are using Tokio. As another poster said not only does the ecosystem fragment along the async/non async lines but along the runtime lines. Async is an extremely leaky abstraction. You are in a way making my point for me. If you want to avoid painful refactoring you should basically always start out tokio-async and shim in non async code as needed because going the other way is going to hurt.

germandiago• 2 years ago

Iterators blew the writer's mind... Most languages have iterators, come on... Java, C#, Python, C++, only to name a few

assbuttbuttass• 2 years ago

I have to say, I am not convinced by this article that async composes better. The nice thing about green threads/fibers is you can make concurrency an internal detail: a function might spawn threads internally, or block, but the caller is free to use it as any other normal function, including passing it to a map() or filter() combinator.

By contrast, async forces the caller to acknowledge that it's not a regular function, and async functions don't compose at all with normal code. You have to write async-only versions of map() filter() and any other combinators.

Maybe async composes better with other async, but with threads, you can just compose with any other existing code.

pornel4• 2 years ago

The big one not mentioned is cancellation. It's very easy to cancel any future. OTOH cancellation with threads is a messy whack-a-mole problem, and a forced thread abort can't be reliable due to a risk of leaving locks locked.

In Rust's async model, it's possible to add timeouts to all futures externally. You don't need every leaf I/O function to support a timeout option, and you don't need to pass that timeout through the entire call stack.

Combined with use of Drop guards (which is Rust's best practice for managing in-progress state), it makes cancellation of even large and complex operations easy and reliable.

jeffbee3• 2 years ago

Cancellation is not worth worrying over in my experience. If an op is no longer useful, then it is good enough if that information eventually becomes visible to whatever function is invoked on behalf of the op, but don't bother checking for it unless you are about to do something very expensive, like start an RPC.

pornel4• 2 years ago

It's been incredibly important for me in both high-traffic network back-ends, as well as in GUI apps.

When writing complex servers it's very problematic to have requests piling up waiting on something unresponsive (some other API, microservice, borked DNS, database having a bad time, etc.). Sometimes clients can be stuck waiting forever, eventually causing the server to run out of file descriptors or RAM. Everything needs timeouts and circuit breakers.

Poorly implemented cancellation that leaves some work running can create pathological situations that eat all CPU and RAM. If some data takes too long to retrieve, and you time out the request without stopping processing, the client will retry, asking for that huge slow thing again, piling up another and another and another huge task that doesn't get cancelled, making the problem worse with each retry.

Often threading is mixed with callbacks for returning results. The un-cancelled callbacks firing after the other part of the application aborted an operation can cause race conditions, by messing up some state or being misattributed to another operation.

aidenn02• 2 years ago

If you implemented async futures, you could have also instead implemented cancelable threads. The problem is fairly isomorphic. System calls are hard, but if you make an identical system call in a thread or an async future, then you have exactly the same cancellation problem.

pornel4• 2 years ago

I don't get your distinction. Async/await is just a syntax sugar on top of standard syscalls and design patterns, so of course it's possible to reimplement it without the syntax sugar.

But when you have a standard futures API and a run-time, you don't have to reinvent it yourself, plus you get a standard interface for composing tasks, instead of each project and library handling completion, cancellation, and timeouts in its own way.

Matthias247• 2 years ago

It's not easy to cancel any future. It's easy to *pretend* to cancel any future. E.g. if you cancel (drop) anything that uses spawn_blocking, it will just continue to run in the background without you being aware of it. If you cancel any async fs operation that is implemented in terms of a threadpool, it will also continue to run.

This all can lead to very hard to understand bugs - e.g. "why does my service fail because a file is still in use, while I'm sure nothing uses the file anymore"

pornel4• 2 years ago

Yes, if you have a blocking thread running then you have to use the classic threaded methods for cancelling it, like periodically checking a boolean. This can compose nicely with Futures if they flip the boolean on Drop.

I’ve also used custom executors that can tolerate long-blocking code in async, and then an occasional yield.await can cancel compute-bound code.

Night_Thastus2• 2 years ago

I don't follow how threads are hard to cancel.

Set some state (like a flag) that all threads have access to.

In their work loop, they check this flag. If it's false, they return instead and the thread is joined. Done.

ric2b• 2 years ago

So you make an HTTP request to some server and it takes 60 seconds to respond.

Externally you set that flag 1 second into the HTTP request. Your program has to wait for 59 seconds before it finally has a chance at cancelling, even though you added a bunch of boilerplate to supposedly make cancellation possible.

naasking• 2 years ago

Async/await have one property that is slightly more difficult to achieve with threads, but isn't really discussed much and hasn't been used much to my knowledge: because context is captured fairly explicitly as records, this makes it easy to persist that context to durable storage and/or migrate them to other machines without any additional runtime machinery.

Achieving this with threads is possible but requires additional runtime support. Basically it requires scanning stack frames, which the garbage collector already does, so you're extending the GC to record stack in a separate place.

cryptonector10• 2 years ago

As well, this functions as state compression by the developer. This is what makes async/await and CPS so much better than thread-per-client: smaller memory footprint!

cryptonector10• 2 years ago

Why choose async/await over threads?

I assume the intended question is: why use async/await over thread-per-client designs?

Easy: because async/await allows you to more easily compress the memory footprint of client/request/work state because there is no large stack to allocate to each client/whatever.

One can still use threads (or processes) with async/await so as to use as many CPUs as possible while still benefiting from the state compression mentioned above.

State compression -or, rather, not exploding the state- is critical for performance considering how slow memory is nowadays.

Async/await, manual continuation passing style (CPS) -- these are great techniques for keeping per-client memory footprint way down. Green threads with small stacks that don't require guard pages to grow are less efficient, but still more efficient than threads.

Threads are inefficient because of the need to allocate them a large stack, and to play MMU games to set up those stacks and their guard pages.

Thread-per-client simply does not scale like async/await and CPS.

the__alchemist• 2 years ago

I am suspicious that Async, slowly taking over the library ecosystems, will be what drives me away from Rust. Over the past year, it's taken over embedded. I was previously worried about excessive use of generics, but this was, gradually over the past ~2 years replaced with Async.

The std-lib HTTP ecosystem has already gone almost full-async. I was able to spin up a simple web server without it using the rouille library recently. I chose this one because it was one of two options that wasn't Async.

I think I may be the only person who A: Enjoys rust for embedded targets (Eg Cortex-M), but doesn't enjoy Async. The easy path is to go with the flow and embrace Async in those domains (Embedded, web backends). I say: No. Threads, interrupts, DMA, etc are easier to write and maintain, and critically, don't split code into two flavors. You don't just add an Async library to your program: It attempts to take your program over. That is the dealbreaker.

littlestymaar24• 2 years ago

The easy path is to go with the flow and embrace Async in those domains (Embedded, web backends). I say: No.

Why are you making a principle of not using async though? It's just one particular feature of the language, there's nothing wrong in not being particularly enthusiast about it, but entirely refusing to use it sounds really weird to me.

Every JavaScript junior use async right now, it's not as it is was a particularly difficult programming paradigm to learn. Sure async in rust is a bit more complicated than in JS due to the fact it's lower level, but it's not worse than anything else in Rust!

Threads, interrupts, DMA, etc are easier to write and maintain,

Not really. I've no doubt that you master them better, but there's no reason to think you won't be able to be as proficient with async. And in fact, async is just strictly more powerful than sync code: you can do exactly what you're doing right now (just write await), plus you get a few features for free (concurrency, cancelation). There's a reason why it's popular actually!

and critically, don't split code into two flavors

In practice, there isn't either. No if you're writing an application, it's only when you're writing a library and you want to support explicitly the use case of people like you that have moral issues with async.

galangalalgol3• 2 years ago

It can be frustrating to: People who target wasm People who target embedded

And superfluous to: People who have compute bound problems

People with one incoming and one outgoing connection at all times.

unstruktured• 2 years ago

To me, implementing an async api using algebraic effects is the end game. Gets rid of the need of an async keyword and no monads required!

mattgreenrocks• 2 years ago

Async/await makes more sense in some languages than others.

For most PLs, algebraic effects look ideal. Just seems like we're figuring out how to talk about them to users and what the UX should be.

debugnik• 2 years ago

Gets rid of the need of an async keyword

You still need an async effect handler though, which is effectively the same syntax wise.

jupp0r• 2 years ago

One of the main benefits of async/await in Rust is that it can work in situations where you don't even have threads or dynamic memory. You can absolutely use it to write very concise code that's waiting on an interrupt on your microcontroller to have read some data coming in over I2C from some buffer. It's a higher level abstraction that allows your code to use concurrency (mostly) without having tons of interactions with the underlying runtime.

Every major piece of software that I have worked on has implemented this in one form or another (even in non-modern C++ where you don't have any coroutine concepts, Apple's grand central dispatch, intel's Thread Building Blocks, etc). If you don't then your business logic will either be very imperformantly block on IO, have a gazillion of threads that make development/debugging a living hell, or be littered with implementation details of the underlying runtime or a combination of all 3.

If you don't use existing abstractions in the language (or through some library), you will end up building them yourselves, which is hard and probably overall inferior to widely used ones (if there are any). I have done so in the past, see https://github.com/goto-opensource/asyncly for C++.

shmerl• 2 years ago

I always understood async/await as expression of concurrency and threads as expression of parallelism. Which aren't the same thing.

Concurrency breaks computation into chunks that can be interleaved even on a single CPU advancing computation concurrently overall but not necessarily physically at the same time for each chunk.

Parallelism on the other hand breaks computation into chunks that literally can run in parallel on different CPUs.

You can even combine both in some fashion.

trevor-e• 2 years ago

As an observer who has used async/await in other languages (Javascript, Swift) I'm really confused why this is such a contentious topic in Rust. swift-concurrency has had its issues as well, but almost everyone I know finds it way more ergonomic and useful than how things were done before. As someone currently learning Rust, is there something particular about the language compared to how Swift did it? Outside of HN I would not know this is such a controversial topic.

anybodyz• 2 years ago

I worked with a fairly large code base which leveraged threads to avoid having to use callbacks in a C++ code base. This allowed the engineers to use the more familiar linear programming style to make blocking network calls which would stall one thread, while others were unblocked to proceed as responses were received. The threads would interoperate with each other using a similar blocking rpc system.

But what ended up happening was that the system would tend to block unnecessarily across all threads, particularly in cross thread communication. These were due to many external network calls being performed sequentially when they could have been performed in parallel, and likewise cross-thread communication constantly blocking due to sequential calls. The end result was a system who'd performance was just fundamentally hobbled by threads that were constantly waiting on other threads or series of external calls, and it was very difficult to understand the nature of the gordian knot of blocking behavior that was causing these issues.

The main problem with using threads is that the moment you introduce a different cpu into the mix you need to deal with synchronization primitives around your state, and this means that engineers will use fewer threads (at least in our case) than necessary to reduce the complexity of the synchronization work needed to be done which means that you lose the advantage of asynchronous parallelism. Or at least that is what happened in this particular case.

The cost of engineering synchronization for async/await is zero, because this parallelism happens on a single thread. Since the cpu "work" to be done for async/io is relatively small, this argues for using single threaded 'callback' style solutions where you maximize the amount of parallelism and decrease the amount of potential blocking as well as minimizing the complexity of thread synchronization as much as possible. In cases where you want to leverage as many cpu's as possible, it's often the case that you can better benefit from cpu parallelism by simply forking your process on multiple cores.

aidenn02• 2 years ago

ELI5: If you have to modify the language anyways (to add async/await) why not just have green threads that wrap NBIO with seemingly blocking calls?

dragonwriter2• 2 years ago

A common refrain is that threads can do everything that async/await can, but simpler

This is the first time I've heard that, and given how many languages have added async/await after threading support, I don't think it is really a broad consensus.

alexxys• 2 years ago

The article talks a lot about async/await but fails to clearly state the main advantage of async code over threads. Async code in general (not only in Rust) allows a server to process thousands of client connections concurrently with minimal latency in a single thread. Even if each client request needs several seconds to process it (assuming the processing is IO-bound). One thread (or more generally, a small number of threads) is much cheaper resource-wise than thousands of threads (in a thread per client scenario).

zac23or• 2 years ago

Async and Thread are of different abstraction levels[1]. The easier to use, the higher the abstraction level.

Rule of thumb: use the highest level of abstraction for your case.

Of course, the Async implementation can be atrocious or in conflict with something already used by your code, forcing you to use a lower level of abstraction.

[1]https://blog.codinghorror.com/the-wrong-level-of-abstraction...

tison• 2 years ago

Days before I'd prefer manipulating between threads manually, because I used to do it for years.

But as it goes much mature the async-await especially with stackless coroutine implementation, writing in async-await is more pleasure than handling threads.

Handmade futures can have some pitfalls, like this blog wrote: https://blog.waynest.com/2022/12/async-cancellation/

However, I don't like Rust's trend of making global, implicit async runtime, especially with #[xxx:main]. Runtime should be specified clearly, like Netty channel must explicitly bind to an EventLoopGroup. And Java's CompletableFuture without specifying a concrete executor causes many issues already.

fennecfoxy2• 2 years ago

Imo, async/await is useful in that it's a simpler approach - developers don't need to worry about threading and all its inherent problems which for the large majority of web and semi-web related tasks is completely fine; the same reason we use languages with GC; no point managing your own memory for most tasks.

Someone writing a driver or something is obviously going to use proper threading and deal with all of the intricacies and gotchas.

Why choose async/await over threads?

Related Stories