Async2 – The .NET Runtime Async experiment concludes

https://steven-giesel.com/blogPost/59752c38-9c99-4641-9853-9cfa97bb2d29

by Nelkins • 2 years ago

137 94 2 years ago

xeromal• 2 years ago

It's a bummer green threads didn't work out

I still remember first hearing about Rust and how it was using Green Threads, then out of the blue a year later passes or months, and I'm reading that it doesn't do any of that anymore, it's basically C++ on steroids mixed with functional programming. I never did look up why they gave up on Green threads and other things, I wonder if they faced similar challenges?

wyager2• 2 years ago

Rust has settled into an excellent design niche where the language does not provide any features that require a language runtime.

Rust's async/await is a purely compile-time feature, with no runtime support required.

This is extremely powerful because it means you can do things like easily run async code on embedded devices with no OS.

To run async code in rust, you bring your own executor. The most popular one is tokio (for desktop OSs), but you also have stuff like pollster (for running async code as blocking code) and embassy or RTIC (for embedded).

jtrueb• 2 years ago

Agreed, as much as there are complaints about usability of async in Rust, many alternatives proposed would fail in resource constrained systems. Green threading would be impossible or very limiting on embedded, where each stack eats away at something like 1-2% of memory and allocation is likely impossible or prohibitively expensive.

I do note that some of the futures are starting to take up a couple KiB of RAM and ROM.

eknkc• 2 years ago

I might be completely wrong as I only played with rust but the current method seems to couple any async libraries you’d like to write to the executor. Which feels wrong imo.

Would it be feasible to provide async io (file, network) code in std and let the executors execute?

Or even some std traits that abstract the filesystem, network etc so that the executors can bring their own implementations but at least library code could only depend on std?

anonymoushn4• 2 years ago

This is a fine approach but it's trivial to ship green threads with no runtime support as well. Various libraries implementing the feature exist for C or Zig etc.

dboreham• 2 years ago

User level (aka green) threading is difficult. Very few implementations work (Erlang, Golang, <any others>+) and it takes a great deal of time and effort to get to that "fully working" state.

+Perhaps the new Java fiber stuff works but I don't have enough data to be sure yet.

Ygg22• 2 years ago

There were plans for green threads, however, green threads were abandoned due to runtime costs around like 6 years ago?

https://github.com/rust-lang/rfcs/blob/master/text/0230-remo...

EDIT: 10 years ago.

steveklabnik2• 2 years ago

It’s coming up on ten years. Time flies.

anonymoushn4• 2 years ago

"green threads" as discussed in TFA don't require a runtime. The relevant issues are maybe https://github.com/rust-lang/rust/issues/33368 and https://internals.rust-lang.org/t/what-is-the-current-safety...

steveklabnik2• 2 years ago

https://github.com/rust-lang/rfcs/blob/master/text/0230-remo...

Note this was in 2014, before Rust 1.0.

hyutmyjahsd• 2 years ago

I miss peak rust.

giancarlostoro3• 2 years ago

Yes it was a while back, wow 10 years.

jasode• 2 years ago

I still remember first hearing about Rust and how it was using Green Threads, [...] I never did look up why they gave up on Green threads

It's because green threads have unavoidable performance costs. I collected various threads that has Rust contributors explaining the costs they didn't want to pay: https://news.ycombinator.com/item?id=39062953

A language like Go is willing to pay those performance costs because they deliberately sit at a higher level of abstraction than lower-level Rust.

Each chose different tradeoffs for different goals:

- Golang : green threads worth the perf cost to gain higher productivity (goroutines, channels, etc) for the type of programs that Go programmers like to write

- Rust : green threads are not worth the perf cost so as to not sacrifice the best possible performance for the type of programs Rust programmers like to write

You can't have a language+runtime that satisfies both camps 100% perfectly. The language designer must choose the tradeoff. But because this tradeoff decision is not blatantly well-known and disseminated ... it also perpetually confuses Language X proponents on why Language Y didn't do the same thing as Language X.

01HNNWZ0MV43FF3• 2 years ago

What do Go channels do that takes advantage of their green threads?

giancarlostoro3• 2 years ago

Thank you for this! This is exactly why I posted my comment, I was hoping someone would share some insight about Rust.

zamalek• 2 years ago

Rust can have green threads. It depends on the reactor. The abstraction sits one level lower than nearly everything else out there.

cube22223• 2 years ago

This all makes sense but

Green threads are different. The memory of a green thread is allocated on the heap. But all of this comes with a cost: As they aren't managed by the OS, they can't take advantage of multiple cores inherently. But for I/O-bound operations, they are a good fit.

this is clearly not true? Am I missing some nuance here, as I'm sure the author knows what they're talking about?

Green threads can totally use a multi-threaded runtime, like e.g. Go does, and it works just fine. The main hurdle with them is arguably FFI.

DougBTX• 2 years ago

The “inherently” means not by default, i.e., the runtime has to support moving green threads between OS threads itself.

cube22223• 2 years ago

Ah, that makes sense, thanks!

01HNNWZ0MV43FF3• 2 years ago

That's not how I use "inherent", maybe they should just say "default" then?

But that's like... C is also single threaded by default, what isn't?

louthy• 2 years ago

Presumably, it just means there needs to be explicit forking of the green thread for cpu bound operations, otherwise everything will run synchronously (because there’s no point where the green thread is paused to wait for an IO IRQ).

That is unless your compiler or JIT injects occasional yields into your synchronous code!

Rohansi• 2 years ago

And that wouldn't be great for performance.

adgjlsfhk1• 2 years ago

Jits already have to do this for GC so it's actually free

bilekas• 2 years ago

There are nuances with multi-threading in C#.

I don't agree with OP about I/O-bound ops, I think if you're looking to green threading, you've taken a wrong approach.

[0] the Task.Runmethod offloads the provided action to the thread pool, and the await keyword yields control back to the caller until the task completes.

All async code must be in an async call stack, virtual threads are 100% transparent because its the runtime scheduling them so you get a but more control than relying on the yeild of dotnet at least as I see it.

Again I don't see the huge demand for it personally, but I barely touch dotnet too often so take this with a grain of salt.

[0] https://stackify.com/c-threading-and-multithreading-a-guide-...

arghwhat• 2 years ago

I don't agree with OP about I/O-bound ops, I think if you're looking to green threading, you've taken a wrong

It depends in the implementation. In Go for example, all I/O is async and suspend your green thread, replacing it with another runnable green thread.

This works the same as if you managed an event loop on your own for the purpose of I/O, which is the best way to handle I/O outside for regular user space code. It’s just automatic with your code resembling a simple, blocking scenario.

OPs note on threading would be C# or runtime specific - green threads have no problem with parallelism, with runtimes commonly having a thread per core (or more) and having them all run green threads in parallel.

neonsunset6• 2 years ago

What this likely means is for you to take advantage of the underlying runtime multiplexing green threads over multiple physical ones running on multiple cores, you need to explicitly fork the execution flow.

This could be as simple as a web server firing off a new green thread or a goroutine for an incoming request, or as contrived as doing so manually within a function scope.

In practice, there really is not much difference with async/await. "Green threads" is a combination of implementation details and a subset of what async/await abstractions achieve.

Effectively, Goroutines are in many ways similar to C# Task<T>s. The difference is that in Go you are expected to explicitly send the result via a channel or some other data structure and then synchronize the completion of the execution, where-as with tasks you simply await that.

There could be an argument made about preference of implicit suspend (Go, Java, BEAM family) over explicit suspend (C#/F#, Rust, JS, Python, C++ co_await, Swift), but for practical purposes invoking a function with 'go' keyword in Golang is very similar to firing off a synchronous method with Task.Run in C#, or calling an asynchronous method (with sufficiently short body before first yield) and not immediately awaiting it.

As I usually post it on HN, tasks make the following patterns trivial:

    using var http = new HttpClient {
        BaseAddress = new("https://news.ycombinator.com/")
    };

    // not immediately awaited requests are executed in parallel
    var frontPage = http.GetStringAsync("news?p=1");
    var secondPage = http.GetStringAsync("news?p=2");

    Console.WriteLine($"{await frontPage}\n\n{await secondPage}");

spinningslate3• 2 years ago

The difference is that in Go you are expected to explicitly send the result via a channel or some other data structure and then synchronize the completion of the execution, where-as with tasks you simply await that.

That may be be the case in Go but it's not an inherent property of green threads. See, for example, Gleam Tasks [0] which are based on green threads and provide the syntatic convenience of being able to await the result rather than receiving a message:

    let task = task.async(fn() { do_some_work() })
    let value = do_some_other_work()
    value + task.await(task, 100)

They do so without the disadvantage of bifurcating the code base into sync and async functions.

[0] https://hexdocs.pm/gleam_otp/gleam/otp/task.html

pron4• 2 years ago

The efficiency and complexity of user mode threads heavily depend on constraints imposed by the particular language. E.g. if the language supports pointers into the stack, user mode threads would be less efficient; if the language is largely dependent on manual memory management -- user mode threads would be more expensive; if the language already has some other concurrency primitives (like async/await) -- user mode threads will be more expensive (although in this case in terms of complexity rather than runtime efficiency). Because Java exposes relatively little of its implementation details, we've been able to implement efficient user mode threads even without any FFI overhead.

andygocke8• 2 years ago

Well, any additional FFI overhead, right?

The cost for exposing very little tends to be that marshaling costs more due to the requirement that values be copied between domains rather than shared.

andygocke8• 2 years ago

I'm not sure that the original description is precisely correct, but yours isn't correct either.

Basically, you can't treat green threads just like "a multi-threaded runtime" and have it just work. That is, a 1:1 mapping between green threads and OS threads is just OS threads.

So fundamentally if you bounce your green stacks off of the actual stack they're going to need to go somewhere... and that place must be the heap.

There are pluses and minuses to this implementation, but the biggest minus is that it makes FFI very complicated. C# has an extremely rich native-interop history (having historically been used to integrate closely with Windows C++ applications) and therefore this approach raised some serious challenges.

In some sense, async is the cost for clean interop with the C/system ABI. Transition across OS threads requires something like async.

cube22223• 2 years ago

I meant that you can have a multi-threaded runtime that will be executing your green threads in a multi-threaded fashion. Like in Go you have (by default) as many worker OS threads as CPUs, and the Go runtime will take care of scheduling your green threads on those worker OS threads (+ create threads as needed for blocking syscalls if I remember correctly, but that's getting way to deep into the details). And this will, in fact, "just work" from the user's perspective.

And yes, as both you said, and I said at the end of my previous comment, the main hurdle of green threads imo is FFI, but it's not what the article mentions, which is what surprised me.

bob10292• 2 years ago

I don't know if this proposal makes a lot of sense.

The existing async1/TPL path is stable & predictable. If you find yourself needing more performance, you can reach for hardware Thread instances and use whatever locking/waiting/sharing/context strategies you desire. Anything else is a weird blend of compromises that is going to have caveats that are not immediately obvious for your specific use case.

For example, async2 w/ runtime JIT appears to have some tradeoffs with regard to GC & memory usage and the experiment writeup leaves some open-ended questions here[0].

[0] https://github.com/dotnet/runtimelab/blob/feature/async2-exp...

whaleofatw2022• 2 years ago

Trying to do clever stuff with threads tends to lead from brow beating, wailing, and gnashing of teeth by various folks who insist to just trust the threadpool.

GordonS2• 2 years ago

If you find yourself needing more performance, you can reach for hardware Thread instances

True for code your team write, but async/await is kind of viral, and many libraries now only have async APIs, which makes them difficult to shoehorn into a threading approach.

bob10292• 2 years ago

async/await is kind of viral

Exactly part of my concern. Hypothetically we've got 2 competing viruses now.

andygocke8• 2 years ago

In practice we don't think there will end up being tradeoffs in async2 vs. async1. If you look below, at the "JIT state machine" section, you'll see that async2 looked better and the GC behavior differences were probably transient.

Overall, there are no architectural reasons why the compiler version should be better. The runtime should be able to make perf decisions that are at least as good in every case.

neonsunset6• 2 years ago

For everyone reading this blog post I caution to take the conclusions there with a grain of salt as they are an interpretation of the notes written down here: https://github.com/dotnet/runtimelab/blob/feature/async2-exp...

It is difficult to draw conclusions at the present moment on e.g. memory consumption until the work on this, which is underway, makes it into mainline runtime. It's important to understand that the experiment was first and foremost a research to look into modernizing async implementation, and was a massive success. Now once that is proven, the tuned and polished implementation will be made.

Once it is done and makes into a release (it could even be as early as .NET 10), then further review will be possible.

With that said, thank you for writing it, .NET tends to be criminally underrated and slept on by the most of the industry, and these help to alleviate it even if just a bit.

hirvi74• 2 years ago

.NET tends to be criminally underrated and slept on by the most of the industry

I have been programming in .Net/C# for about 8 years now. I absolutely hate Microsoft with every fiber of my being. However, I can't thank them enough for C# (and all the FOSS contributors as well). .Net has been such a pleasant experience that I truly do not want to program in any other language.

naasking3• 2 years ago

I absolutely hate Microsoft with every fiber of my being.

Why? I get it back when they were super hostile to Linux and open source, but those days are long past.

runevault2• 2 years ago

I see people complain sometimes that c# adds too many features, and I understand the concern, but with limited exceptions (I don't like the new global using stuff), they tend to feel tasteful to me. Also I'm really hopeful we'll get Discriminated Unions (under a name I'm forgetting) in c# 10.

leosanchez• 2 years ago

It's C# 12 now. Maybe we will get discriminated unions in C# 15 or so

jayd165• 2 years ago

C# is a multi-paradigm language with everything and the kitchen sink but somehow the syntax and elegance is still best in class.

ikekkdcjkfke3• 2 years ago

What is the fundemental problem we are trying to solve here? My pet problem with async is that it takes so much syntax, and i wonder why we need to do that. As i understand it it is all about blocking calls taking up OS threads.

So i will try to naievly solve it here, and maybe i end up with the same conclusions.

When doing a blocking call, the OS thread could just start executing something else transparently and not 'yield' anything to the code, it just stops executing the code and comes back and executes further, no await, no tasks no nothing. Ok, but how did we end up in a thread? The http listener started 4 threads and it's just putting a stack of function pointers and context memory for the threads to eat when they want. There is a separate engine on another OS thread that handles where the threads can start executing ready code again. No Task or await keywords show up in this code. I have no idea how stack traces work but i guess that can just be saved to memory and loaded back again when the threads feel like executing some ready code

irdc2• 2 years ago

The rationale for async/await I keep hearing is that the separate stack required for each thread uses a lot of memory. I’m not sure going full on function colouring is the answer though. A concerted effort for language runtimes to use less stack space and then simply allocating smaller thread stacks sounds to me like a more elegant solution. It certainly is a whole lot easier to debug than a deeply-nested async/await chain.

iknowstuff• 2 years ago

But also avoiding the cost of context switching, and the ability to handle many tasks in embedded environments.

https://tweedegolf.nl/en/blog/65/async-rust-vs-rtos-showdown

layer82• 2 years ago

What you propose is similar to how Java virtual threads work.

ikekkdcjkfke3• 2 years ago

Thanks for pointing me in right direction. That's exactly what I'm wishing for in .net

jayd165• 2 years ago

Ok well OS threads are already scheduled this way... The drawback is they're heavier because they make assumptions that the runtime doesn't need to make.

Furthermore, explicit Async/await provides a syntax for co-opoerative multithreading and that enables other patterns that implicit designs don't.

kodablah• 2 years ago

And for the transition phase, there has to be interop for async ↔ async2

For those like me who weren't clear whether `async2` was expected to be a real keyword in the final language, it's not[0].

0 - https://github.com/dotnet/runtime/issues/94620#issuecomment-...

augusto-moura• 2 years ago

async2 looks like such a terrible keyword name, that it didn't even crossed my mind as an option. Only following your linked comment I understood that they used it as a temporary name while testing

TwentyPosts• 2 years ago

Have you read the article?

As async is already used as an identifier in C#, the team decided to use async2 as a codename for the experiment. If that thing ever makes it into the runtime, it will be called async - so it will be a replacement for the current async implementation.

jayd165• 2 years ago

I'm encouraged by the talk of improved exception handling and a recognition of how much garbage it creates. It makes it hard to do things in an idiomatic way and keep GC pressure low.

I hope they also revisit task state tracking and TaskCanceledException. It feels out of place to use an exception (and unroll the stack) as control flow in the same API they added ValueTask to keep Task handles on the stack.

Async2 – The .NET Runtime Async experiment concludes

Related Stories