Local async executors and why they should be the default

https://maciej.codes/2022-06-09-local-async.html

by rklaehn • 3 years ago

220 306 3 years ago

maciej.codes

evanrelf• 3 years ago

Posted on June 9, 2022

rklaehn19• 3 years ago

Yeah, it is old. But I only found it recently and did not see a discussion of it yet.

I found this quite interesting. I love rust, but async rust always seems almost like a different language.

Lifetimes become much more complex and much less useful, you use Box and Arc way too much. It seems like you would be better off if everything was heap allocated.

Working with local futures brings back some of the things I love about rust, but it is certainly not prominently featured in libraries like tokio and the entire async ecosystem.

blowski• 3 years ago

posted on June 9, 2022
this is old

???

mgaunard4• 3 years ago

Multithreading, why does everyone always do it wrong?

One of life's big questions.

worldsavior• 3 years ago

Because there are no actual good sources to implement async/multi threading programs.

rklaehn19• 3 years ago

I think rust does it right in many cases. Raw rust multithreading is a pleasure to use, and some libraries like rayon are extremely unobtrusive.

That is why I am somewhat unhappy with the state of async rust. It does not yet have the quality and unobtrusiveness of sync rust libraries like rayon.

buildbot• 3 years ago

Yeah, I just recently picked up rust and rayon is so awesome. It is great to be able to instantly parallelize an iterator.

junon• 3 years ago

Pray tell, what's the objective right way?

monsieurbanana• 3 years ago

He's saying nobody has found it yet

darthrupert• 3 years ago

99% of the cases: don't. 1%: very carefully.

Thoeu388• 3 years ago

Redis nailed multithreading :)

samsquire4• 3 years ago

CPU cycles are practically free but memory latency and synchronization are not. (Hardware threads gives more CPU cycles capacity). People's scalability problems are memory related, not CPU cycles. In other words, we can't scale updating a single memory location with more hardware threads. People often want to scale the updating of something so they add threads, but the contention of memory synchronization prevents that.

Scaling independent memory updates works well and is embarrassingly/pleasingly parallel.

I am working on a multithreaded design that is scalable and easier.

The idea is that if you represent your problem, data flow and control flow as a tree where branches do not communicate, you can get the purported scalability of multiple hardware threads.

mgaunard4• 3 years ago

You realize people have been doing those things since the 70s?

weinzierl2• 3 years ago

"If you write regular synchronous Rust code, unless you have a really good reason, you don't just start with a thread-pool. You write single-threaded code until you find a place where threads can help you, and then you parallelize it, [..]"

I cannot agree more with that. As someone who's done a good deal of Java in my day job, I can tell you a thing or two about spawning threads willy-nilly. At least it is easier to avoid in Rust, but I'd still prefer it the other way round: opt-in, instead of opt-out.

thinkharderdev• 3 years ago

It is opt-in. If you're using Tokio then you can specify whether you want to use a single-threaded or multi-threaded runtime. Multi-threaded is "default" in the sense that if you just use `#[tokio::main]` then you get a multi-threaded runtime but you can also just do `#[tokio::main(flavor = "current_thread")]` to get a single threaded executor.

More to the point, even using a multi-threaded runtime won't spawn threads willy-nilly. It will default to using N worker threads (where N is the number of CPU cores available).

exomancer2• 3 years ago

It is opt-in. If you're using Tokio then you can specify whether you want to use a single-threaded or multi-threaded runtime. Multi-threaded is "default" in the sense that if you just use `#[tokio::main]` then you get a multi-threaded runtime but you can also just do `#[tokio::main(flavor = "current_thread")]` to get a single threaded executor.

Doing something extra to get a behavior different than default is the definition of opt-out my friend.

weinzierl2• 3 years ago

The willy-nilly was in regard to Java and more precisely the ecosystem, not the language itself. In Java I find it hard to have N worker threads, where N is the number of CPU cores available, because many common libraries spawn threads - willy-nilly I'd say.

I have not enough experience about the Rust ecosystem but I hope it will successfully avoid a similar fate.

resonious• 3 years ago

Yes the RwLock and mpsc comes from Tokio and lets you .await instead of blocking a thread, but these are not async primitives, these are multi-threading synchronization primitives.

The only reason all this async stuff even exists is because we want concurrency. We want to say "while this one task waits for I/O, this other task will do stuff". So it's not too surprising to me that an intro to async would include synchronization primitives. Those primitives aren't really "thread"-specific if by thread you mean OS thread. When you do async like this, you're basically re-implementing OS threads in user space.

hinkley• 3 years ago

Async/await is green threads with better scoping and syntactic sugar.

Which lets be honest, is something green threads desperately needed.

duped7• 3 years ago

They're not green threads. Futures are stackless coroutines.

foldr2• 3 years ago

I think the article's point is that if you're doing single-threaded concurrency then you don't (or at least might not) need synchronization primitives. If you're writing a Node script, for example, then you don't have to worry about two callbacks trying to modify the same variable at the same time because you know that there is only one thread of execution.

rklaehn19• 3 years ago

Multi threading and concurrency are not the same. You can have a very high performance server that handles thousands of requests concurrently on a single thread. That's how node/deno do things.

But the way to do things in async rust is that if you want concurrency you also have to use multithreading. At least that is what you see in all the examples and docs.

As soon as you require your futures to be Send you have to use not just concurrency but full blown multithreading primitives.

So e.g. you have to use Arc<RwLock<...>> where a Rc<RefCell<...>> would suffice.

Now, to be fair, both futures and tokio support local futures. There is boxed_local (https://docs.rs/futures/latest/futures/future/trait.FutureEx...) and tokio-util even comes with an optional local task pool (https://docs.rs/tokio-util/latest/tokio_util/task/struct.Loc...).

But these features are hidden and certainly not promoted in examples and documentation.

menaerus3• 3 years ago

So e.g. you have to use Arc<RwLock<...>>

In terms of C++ code that would equate to std::shared_ptr<std::shared_mutex<...>> which ... sounds quite wasteful in terms of scalability/performance. Why is it not possible to simply return a Rust promise? That's the way I do it in my C++ async (executor) library backed by work-stealing queues under the hood.

VWWHFSfQ• 3 years ago

I believe you can just annotate your tokio::main with flavor = "current_thread"

mgaunard4• 3 years ago

Waiting for I/O does not require using multiple threads. You can have the same thread do other things until I/O is ready.

zokier5• 3 years ago

While the article mostly focuses on the cognitive cost, which I deeply sympathize with, I do wonder about the runtime performance cost. Are there any good benchmarks actually quantifying the impact of all that extra thread-safety and the hoops that it adds? I'm not asking simply due personal interest in seeing the numbers, but also because if we want to steer the community towards this non-threadsafe direction it would help to have material to back the ideas and I suspect Rust community would be more responsive to complaints about perf than cognitive cost.

rklaehn19• 3 years ago

Anecdotal evidence, but many large rust async code bases perform significantly better if you are using a single threaded runtime vs. a multithreaded runtime.

Here is an issue for the quinn crate that implements QUIC: https://github.com/quinn-rs/quinn/issues/1433

We have had very similar experiences when developing https://github.com/n0-computer/iroh

And this is with quinn still carrying around the synchronization primitives to make everything Send.

Another piece of anecdotal evidence: the functional collection library https://github.com/bodil/im-rs comes in two flavours: one using Arc and one using Rc. The reason for this is that you get a significant performance increase by using Rc (where manipulating the refcount is just an integer inc/dec) vs Arc where manipulating the refcount needs to be an atomic op.

Last but not least - often making futures Send requires you to box your futures. Send and lifetimes just does not play well together. Do something like this:

  trait Foo {
    type OpFut<'a>: Future<Output = ()> + 'a;
    fn op(&self) -> Self::OpFut<'_>
  }

This is what eventually you would want something like async_trait to desugar to. Use the above in a Send context, and you get a completely undecipherable higher-ranked lifetime error. So you are back to boxing all futures.

Nullabillity3• 3 years ago

Last but not least - often making futures Send requires you to box your futures. Send and lifetimes just does not play well together. Do something like this:

You should ~never need to box your futures to make them Send. You do sometimes need to box them to make them Unpin, but most of the time this is a tradeoff where it's possible to find a way to avoid moving them instead.

It seems like these screeds (both the OP and this reply) come from the misunderstanding that "X is hard, Y sometimes comes up in the errors when trying to do X, therefore X would be easy if we got rid of Y". In reality, Y often just demonstrates intricacies of X that you weren't thinking about.

duped7• 3 years ago

Send and lifetimes just does not play well together.

That's the point!

yencabulator6• 3 years ago

You might be interested in these. tl;dr People are literally reimplementing tokio (differently) just to avoid this cost.

https://www.youtube.com/watch?v=PbgTyCSDPrs

https://itnext.io/modern-storage-is-plenty-fast-it-is-the-ap... https://news.ycombinator.com/item?id=25220892

https://github.com/DataDog/glommio https://www.datadoghq.com/blog/engineering/introducing-glomm...

runiq• 3 years ago

This is bad editorializing. You're putting words in the author's mouth that cannot even be found on that page.

Don't do that.

Edit: Thanks to the mods or whoever fixed it.

beltsazar2• 3 years ago

I wonder if HN should have a rule that prohibits editorializing.

dvt2• 3 years ago

Not sure if this is facetious, but HN does have a rule that prohibits editorializing. From the Guidelines: "Otherwise please use the original title, unless it is misleading or linkbait; don't editorialize."

voxelghost• 3 years ago

Maybe HN should auto-pull title from URL, and then add the posters title as a form of "subtitle/comment"

pindab0ter2• 3 years ago

https://news.ycombinator.com/newsguidelines.html

distcs3• 3 years ago

If anyone is confused what the "bad editorializing" is ...

Actual title of article: Local Async Executors and Why They Should be the Default

The title posted to HN: Async rust – are we doing it all wrong?

The title has been fixed now. But yeah it is annoying to see badly editorialized titles. It is against the guidelines -- https://news.ycombinator.com/newsguidelines.html

Otherwise please use the original title, unless it is misleading or linkbait; don't editorialize.

hoangnguyenvu• 3 years ago

reason?

datadeft3• 3 years ago

For me the biggest issue with Async is the management of multiple dependent async calls. It has some weird thing going on and I am not sure which pattern to use exactly. Some functions expect exactly same async fn signature some not and I am not sure why and which one to use.

duped7• 3 years ago

I'm confused what you mean here.

If you have a function that "depends on" another function you call it within the other function. If it's async you .await it. Do you mean something about spawning tasks or passing callbacks around?

datadeft3• 3 years ago

I have this function that I still struggle to write.

  async fn function_handler(_event: Request) -> Result<Response<Body>, Error> {
    let aws_clients = AwsClients {
        s3_client: S3_CLIENT.get().await.to_owned(),
        glue_client: GLUE_CLIENT.get().await.to_owned(),
        athena_client: ATHENA_CLIENT.get().await.to_owned(),
    };

    println!("{}", aws_clients);

    let config = CONFIG.get().await;
    let version = VERSION.get().await;

    let db_eid_cache = create_db_eid_cache(aws_clients.clone(), config);
    let table_eid_cache = create_table_eid_cache(aws_clients.clone(), config);
    let job_eid_cache = create_job_eid_cache(aws_clients.clone(), config);
    let table_summaries_cache = create_table_summaries_cache(aws_clients.clone(), config);
    let job_summaries_cache = create_job_summaries_cache(aws_clients.clone(), config);
    let query_summaries_cache = create_query_summaries_cache(aws_clients, config);

    let rez = tokio::join!(
        db_eid_cache,
        table_eid_cache,
        job_eid_cache,
        table_summaries_cache,
        job_summaries_cache,
        query_summaries_cache,
    );

    let six_pack = SixPack {
        db_eid_cache: rez.0,
        table_eid_cache: rez.1,
        job_eid_cache: rez.2,
        table_summaries_cache: rez.3,
        job_summaries_cache: rez.4,
        query_summaries_cache: rez.5,
    };

    let lambda_response = LambdaResponse {
        config,
        result: &six_pack,
        version: &version,
        now: utc_now(),
    };

    let response_body = serde_json::to_string(&lambda_response)?;

    Ok(Response::builder()
        .status(200)
        .header("content-type", "application/json")
        .body(response_body.into())
        .map_err(Box::new)?)

}

Not sure how to do error handling properly or what would be the idiomatic way to implement it.

fsckboy• 3 years ago

the title should be "Local Async Executors and Why They Should be the Default (Rust)"

ikekkdcjkfke4• 3 years ago

In C# i always wondered why they couldn't hide the async/await logic for most cases. I never need to fire off two IO futures at the same time, so just make the thread do other stuff if i'm waiting for IO feedback, don't make me type out async/await in all impacted functions, let the compiler figure out when it can process other stuff

peheje2• 3 years ago

the use of async and await is a design decision that requires knowledge of the program's logic and desired behavior. It's not just a matter of compiler optimization, and that's why the compiler can't automatically figure out where to use these keywords.

Suppose we have a service where users place orders, and we need to:

1. Save the order. 2. Deduct items from inventory. 3. Send a confirmation email.

If we perform these operations asynchronously but sequentially using `await`:

```csharp public async Task PlaceOrder(Order order) { await SaveOrder(order); await DeductItems(order); await SendConfirmationEmail(order); } ```

They happen in the correct order, which is important.

But, if the compiler removed the `await`, causing all three operations to start at the same time:

```csharp public async Task PlaceOrder(Order order) { SaveOrder(order); DeductItems(order); SendConfirmationEmail(order); } ```

This could lead to issues like sending the confirmation email before the order is saved, showing the importance of `await`. The compiler cannot optimize this without understanding the business logic.

ikekkdcjkfke4• 3 years ago

I was thinking more in terms of making good old 'blocking' calls and that the compiler can 'taskify' those old blocking calls to let the thread do some other task instead of waiting on the 'blocking'

gpderetta6• 3 years ago

You have it backward. The compiler should implicitly add the awaits for waitable objects, unless an operation is explicitly async. So you would write (in pseudocode):

  Task PlaceOrder(Order order) { SaveOrder(order); DeductItems(order); SendConfirmationEmail(order); }

And the compiler will implicitly await all three operations (and ideally infer that your function is async).

If you want to overlap computation, you avoid the implicit wait with async:

  Task PlaceOrders(Order order1, Order order2) { 
       let done1 = async PlaceOrder(order1); // async prevents implicit waiting
       let done2 = async PlaceOrder(order2);
       wait_all(done1, done2); // wait_all is also async and implicitly awaited. Ideally this should happen automatically for all unwaited and not returned futures at end of scope
   }

This allows being polymorphic on the async-ness of the function (pardon the pseudo c++):

   template<Range R, callable<R::value_type> F >
   void for_each(R range, F f)  {
       for (auto x : range) f(x); // f(x) is awaited if f is async and for_each itself becomes async.
   }

edit: sometimes it is important that no preemption happens in a region [1], so some scoped marker (atomic { ... } for example) would case a compilation error if an await would be introduced automatically.

edit2: and of course you should be able to use async even if the called function is a boring old blocking one. The runtime can spawn background task (or better yet use work-stealing) to run it.

[1] personally I think that atomicity guarantees should be about data, not code, but whatever.

dpc_012342• 3 years ago

Async is and probably will always be less usable than blocking Rust. It is a very, very useful mode of operating when you really need two of its biggest benefits: lightweight cooperative concurrency and task cancellation, but it comes at a big usability cost.

Rust software should use async tactically - in places where it is needed. Unfortunately handling http, which is a large part of many applications is actually a place where async has benefits. But if you plan to run your http behind nginx anyway (for TLS termination) even there using blocking http server might be a good idea.

If you write regular synchronous Rust code, unless you have a really good reason, you don't just start with a thread-pool. You write single-threaded code until you find a place where threads can help you, and then you parallelize it,

I disagree with this one. When you work on a software project you should have the basic architecture figured out already, and main part of that is breaking your software into structurally parallel parts that can work independently. Adding ad-hoc parallelism after the fact works only for small scale things and will lead to rather accidental concurrency architecture.

Then for each part (groups of threads), figure out if it *needs* async. Between each part you'd communicate via channels or some shared data structures that rather easily can be made to work with both async/blocking code.

So e.g. an async http server, benefits from lightweight async concurrency, makes rpc-like channel-based calls to blocking IO / CPU/bussiness-logic intense workers (that don't benefit from async) where it makes sense. Each part is written in the best "type of Rust for its use-case".

Or if you need ability to cancel certain computations inside the larger framework (e.g. simulating agents etc.) you might want to nest async executor inside a blocking code.

Note: there's a lot of types of program archetypes out there (CRUD, ETL, data-intensive, embedded, frontent SPA, native mobile app) and I've noticed that many people are boxed in the type they happen to work on. CRUD applications (which are very common) are often 90% http handling-based and it might make sense to write them whole in async Rust.

sanxiyn3• 3 years ago

Your CRUD web application server almost certainly doesn't need async Rust. Using a blocking HTTP server is not "might be a good idea", it simply is a good idea.

I recommend Rouille for this: https://github.com/tomaka/rouille. In case you are worried about performance, check the benchmark. Blocking Rouille is faster than builtin async server in Node.js.

hardware2win• 3 years ago

Your CRUD web application server almost certainly doesn't need async Rust. Using a blocking HTTP server is not "might be a good idea", it simply is a good idea.

How so? By what logic?

Blocking Rouille is faster than builtin async server in Node.js

It isnt proof

cmrdporcupine7• 3 years ago

Thanks for this reference. Will bookmark for if I ever need an HTTP service.

Wish this also provided websockets.

EDIT: it does https://github.com/tomaka/rouille/blob/master/src/websocket/...

Looks a bit neglected though. Not much recent development.

dpc_012342• 3 years ago

Yes, but since Rust ecosystem went all in on async http, you'll have to use some niche http framework, and dumb CRUD code is not going to suffer too bad from async's limitations.

I really wish there was a popular and community embraced blocking http framework in Rust, to tip the scales here a bit.

bruce343434• 3 years ago

lightweight cooperative concurrency

It's not cooperative if multiple things happen _at the same time_ which is always touted of rust async. Cooperative would be iterators, or generators, or coroutines/continuations. They let you do things in a single thread but have the execution order be mixed. That's concurrent, but not parallel. What you are talking about is parallel execution. That changes the classification away from cooperative. Sorry, just a pet peeve of mine.

rklaehn19• 3 years ago

If you use rust async with a local executor such as the tokio current thread runtime, it provides lightweight cooperative concurrency.

There is nothing ever happening at the same time, since you are on a single thread. That is why you don't need synchronization primitives such as Mutex but can live with something lightweight like RefCell. And that is why you can get by with non atomic reference counting smart pointers (Rc instead of Arc).

And it is cooperative in that you have to yield by calling await, otherwise nothing else will run.

What the article argues is that the option concurrent but not parallel, which both tokio and futures support in principle, should be advertised more and maybe even be the default.

lionkor• 3 years ago

I find all the async stuff in Rust incredibly ugly, cumbersome, and its one of the biggest reasons I prefer C++, still. C++ lets me just write single- or multithreaded code, because none of the dependencies force their `async` stuff on me. Yeah, its up to me to ensure things are synchronized, but I'd rather do that than try to figure out how to get some dependency that isnt meant to use async to work in some async move closure.

zokier5• 3 years ago

Your dependencies are what you choose them to be, you are not forced to use async libraries if you don't want to.

klabb3• 3 years ago

Technically true, but in practice you’re constrained by whatever is provided by the ecosystem. And the truth is that many Rust projects were forced to choose async or sync, so these are often disjoint and fragmented. Even within async, crate authors need to pick and choose which async ecosystem to support, due to lack of support in std for spawning tasks and running simple IO, which most projects need. This is slightly alleviated by the Tokio hegemony, but that itself is a deferred migration minefield for Rust to handle.

sanxiyn3• 3 years ago

Eh, in that case, you can also just write single or multithreaded Rust code, entirely ignoring async Rust. I do. And multithreaded Rust is much more pleasant experience than multithreaded C++.

Which dependency is forcing async Rust on you? Alternatives are usually available.

borrow• 3 years ago

Postgres (which is a wrapper around tokio-postgres, and as a result drags the whole tokio bloated dependency graph). I would love to have a purely sync alternative to that.

rklaehn19• 3 years ago

I can see where you are coming from. But you can do most things purely in sync rust.

There are sync rust multithreading libs like rayon that are a joy to use, and there are even blocking versions of popular http libraries like reqwest: https://docs.rs/reqwest/latest/reqwest/blocking/index.html

They are usually doing some ugly stuff internally to make this work, but as a pure library user you don't have to care.

If you want to write something small that e.g. pulls some data via http, performs some computation, then pushes the result, it is totally possible to do this fully in sync rust.

And if you are writing a highly concurrent web server, looking into async might not be the worst idea.

eptcyka• 3 years ago

Having written a lot of async and sync rust, and having both components interact with one another, calling async stuff from a sync context isn't even that ugly these days. But maybe it's stockholm syndrome.

bheadmaster4• 3 years ago

One thing I hate about "async" systems in general is the absurdity of suddenly having two types of functions behave differently with same syntax. And you have to add "await" keyword to async functions to make them synchoronous, i.e. behave normally.

I find the Go's approach of "everything is synchronous, except when made asynchronous with the 'go' keyword" much more pleasant and much less error prone. Is there a Rust library that does this - that provides coroutine-style asychronous behavior, but with synchronous Go-like syntax?

vbezhenar• 3 years ago

Can you expand what ugly stuff they're doing internally? I mean, synchronous code is the most simple way to do anything. POSIX API is synchronous by default. Why would you need ugly stuff? Asynchronous API might force you to do ugly stuff (e.g. for files or DNS).

littlestymaar2• 3 years ago

but I'd rather do that than try to figure out how to get some dependency that isnt meant to use async to work in some async move closure

tokio::spawn_blocking[1], see it's not that hard. But sure to use another language you must learn it, an it requires a bit if effort…

[1] assuming you want to use tokio like post people do, but other executors should have the same kind of functions to do that as well if need be.

rklaehn19• 3 years ago

It's not always that simple. What if your sync lib has some callbacks, and you want to do something in the callbacks that requires async.

You could argue that this mismatch exists even with rust itself, where you have Drop, which is sync, and you might have to do something in your drop that requires async, like closing a network connection.

You have to pass in some runtime handle that you can use to spawn a task to do what you have to do. This is definitely not simple and beginner friendly. Or even worse - say goodbye to RAII and tell people that they have to explicitly call an async shutdown fn.

cmrdporcupine7• 3 years ago

You've made the giant assumption that most people working in Rust are working with some kind of executor or framework, and you're either wrong, or the state of the Rust world has become very depressing.

tokio::spawn_blocking doesn't exist in my codebase because I'm not using tokio, nor should I have to in order to use library code from a crate.

This is the problem with async in a nutshell; it's viral, and it brings with it a lot of lifestyle assumptions. Assumptions that I think people who are used to working in the web framework world are maybe comfortable with, but which should not have been allowed to spread to the crates ecosystem broadly.

async is fine as an implementation option for your service or binary; but its propagation into library code means now people using that library are tied into requiring a framework to host it with. And this is frankly just bad hygiene, and in a way seems contrary to Rust philosophy generally.

Most crate writers are polite enough to offer async and non-async bundling of their code. But on more than one occasion recently I've run into dependencies that did not do this, or did not do this thoroughly (offering sync versions with less features, for example).

cmrdporcupine7• 3 years ago

I find the async stuff obnoxious, too, and I avoid it. Helps that I don't work in the web space.

But in general the dependency story with Rust is better than with C++. I don't miss the integration story with C++; just bringing in a dep in the first place is a roll of the dice on whether it's going to work with your build system. And then whether it brings with it some other lifestyle assumptions (exceptions, some third party deps you don't want or can't link to, work with your compiler or not, etc.)

Honestly, I work in systems & embedded stuff, and it's not hard to avoid async in Rust. It's only once you start poking at web-adjacent stuff that you run into the wall, and then when you do you find yourself covered in the taint of people who brought their NodeJS Stockholm Syndrome over to Rust, but that's another rant...

pjmlp20• 3 years ago

Or just stick with cmake + vcpkg/conan, and enjoy the existing ecosystem of C++ libraries.

Nowadays it is impensable to do a Java project without Maven/Gradle, yet it took about 10 years for Ant to be relevant (counting from 1996), and couple more for Maven, and yet another few for Gradle.

Similarly with NuGET and MSBuild evolution.

Yet they were eventually adopted, same is happening with vcpkg/conan.

distcs3• 3 years ago

Can a mod fix the title please? The poster of this story has editorialized the title so bad that it has no connection with the actual title.

Actual title: Local Async Executors and Why They Should be the Default

Posted title: Async rust – are we doing it all wrong?

Really? Why this kind of terrible editorializing?

foldr2• 3 years ago

I agree that this could be less editorialized, but the actual title doesn't include 'Rust', so that needs to be added at least.

distcs3• 3 years ago

Agreed. Perhaps we can have a syntax to declare what is editorialized. We already add "(YEAR)" and "[pdf]" and "[video]". We could "[Rust]" or other keywords like that about the post missing from the title?

dvt2• 3 years ago

If you know anything about asynchronous sockets it should be that multi-threading a socket doesn't actually yield you more requests / second, and it can actually lower it...

Re-read this a few times, and I'm fairly convinced it is not generally true. The author is also being a bit confusing about what exactly he means by "socket" here. Because while it's true that multi-threading over a server socket (e.g. the one that binds to the port when you launch the server) will not yield performance gains, multi-threading clients (that have their own sockets, including file descriptors) definitely will. That's the whole point of nginx thread pools[1]. Note that nginx does zero "CPU-bound" work, it literally just serves files.

Node/Deno being single-threaded is purely a limitation of Javascript. Tomcat, Jetty, etc. are all multi-threaded. I'm a bit tired, so I can't comment on the rest of the post in detail, but this was a bit of a red flag.

[1] https://gist.github.com/denji/8359866

gpderetta6• 3 years ago

Having multiple threads calling 'accept' itself can be a win and Linux (and epoll) has explicit support for it.

namjh2• 3 years ago

Making things thread safe for runtime-agnostic utilities like WebSocket is yet another price we pay for making everything multi-threaded by default. The standard way of doing what I'm doing in my code above would be to spawn one of the loops on a separate background task, which could land on a separate thread, meaning we must do all that synchronization to manage reading and writing to a socket from different threads for no good reason.

Why so? Libraries like quinn[1] define "no IO" crate to define runtime-agnostic protocol implementation. In this way we won't suffer by forcing ourselves using synchronization primitives.

Also, IMO it's relatively easy to use Send-bounded future in non-Send(i.o.w. single-threaded) runtime environment, but it's almost impossible to do opposite. Ecosystem users can freely use single threaded async runtime, but ecosystem providers should not. If you want every users to only use single threaded runtime, it's a major loss for the Rust ecosystem.

Typechecked Send/Sync bounds are one of the holy grails that Rust provides. Albeit it's overkill to use multithreaded async runtimes for most users, we should not abandon them because it opens an opportunity for high-end users who might seek Rust for their high-performance backends.

[1] https://github.com/quinn-rs/quinn

exomancer2• 3 years ago

Also, IMO it's relatively easy to use Send-bounded future in non-Send(i.o.w. single-threaded) runtime environment, but it's almost impossible to do opposite. Ecosystem users can freely use single threaded async runtime, but ecosystem providers should not.

We have Send and non-Send primitives in Rust for a reason. You could use Arc/Mutex/AtomicUsize/... everywhere on a single thread, but you should use Rc/RefCell/Cell<usize>/... instead whenever possible since those are just cheaper. The problem is that in the ecosystem we are building the prevailing assumption is that anything async must also be Send, which means we end up using Send primitives even in non-Send contexts, which is always a waste.

If you want every users to only use single threaded runtime, it's a major loss for the Rust ecosystem.

Running single threaded executors does not prohibit you from using threads, it just depends on how you want to do that. You can:

1. Have a single async executor running a threadpool that requires _everything_ to be Send. 2. Have a single threadpool, each thread running its own async executor, in which case only stuff that crosses thread boundaries needs to be Send.

The argument is that there are many scenarios where 2 is the optimal solution, both for performance and developer experience, but the ecosystem does not support it well.

rklaehn19• 3 years ago

There is quite some support for the second option. But it is not advertised at all.

E.g. there is an entire local task pool impl in tokio_util, but it is not enabled by default and at least for me was difficult to find. https://docs.rs/tokio-util/latest/tokio_util/task/struct.Loc...

I wrote an entire local executor pool before finding that thing and ripping my code out again...

At the very least the support for tasks for non-Send futures should be advertized more, and maybe also the APIs polished a bit.

namjh2• 3 years ago

The problem is that in the ecosystem we are building the prevailing assumption is that anything async must also be Send, which means we end up using Send primitives even in non-Send contexts, which is always a waste.

Honestly I don't think every user want top notch throughput when writing asynchronous Rust applications. Most users would just want the correctness of the Rust type system, and its "lightweight" runtime characteristics compared to CPython, Node.js etc., which can provide fairly good performance.

The thing is, using Arc in single threaded runtime does not greatly harm performance. If it matters, you should be handling 1M+ rps per core and using multithreaded runtime because it scales better, which benefits from using threadsafe primitives.

samsquire4• 3 years ago

This article is against multithreaded executors by default and that synchronous code is easier to read and more practical than async code. I completely understand this.

My hobby and main interest is multithreading, async, coroutines, parallelism so I love articles like this so thank you.

I am trying to design a solution that lets us have our cake and eat it to. I want multithreaded coroutines or multithreaded async executors by default. I am trying to design a server and runtime that is largely parallel, concurrent, efficient, easy to understand, easy to reason about, async and easy to read and maintain. I want:

* Tokio is not bad but I think a codebase that uses async rust requires a high level of skill, cognitive load and understanding. It's not straightforward!

* synchronous straight-line flow code is the easiest to read and follow

* promises and callbacks aren't easy to read or follow control flow

* making a single threaded program parallel after writing it is almost a complete rewrite

* work stealing thread pools solve the starvation problem, so they're good!

* IO shouldn't block CPU and CPU shouldn't block IO

* I am trying to design a syntax that is data orientated that means programs can be parallelised after writing them, so parallelisation comes for free

* we can use the LMAX Disruptor pattern for efficient cross-thread communication. I use a lockfree multiconsumer multiproducer ringbuffer in my programs.

I have an epoll-server which multiplexes clients/sockets over threads, this is more efficient than a thread-per-socket/client. I need to change it into a websocket server.

Imagine you're a search engine company and you want to index links between URLs. How would you model this with async rust and thread pools?

  task download-url
   for url in urls:
    download(url)

  task extract-links
   parsed = parse(document)
   return parsed

  task fetch-links
   for link in document.query("a")
    return link

  task save-data
   db.save(url, link)

How would you do control flow and scheduling and parallelism and async efficiently with this code?

`db.save()`, `download()` are IO intensive whereas `document.query("a")` and `parse` is CPU intensive.

I think its work diagram looks like this: https://github.com/samsquire/dream-programming-language/blob...

I've tried to design a multithreaded architecture that is scalable which combines lightweight threads + thread pools for work + control threads for IO epoll or liburing loops:

Here's the high level diagram:

https://github.com/samsquire/ideas5/blob/main/NonblockingRun...

The secret is modelling control flow as a data flow problem and having a simple but efficient scheduler.

I wrote about schedulers here and binpacking work into time:

https://github.com/samsquire/ideas4#196-binpacking-work-into...

I also have a 1:M:N lightweight thread scheduler/multiplexer:

https://github.com/samsquire/preemptible-thread

binary1323• 3 years ago

Rust acolytes need to come to terms with the fact that if they want Rust to be the next C++, it's going to be the next C++. Heck, even C++ async is simpler. Anyway, who needs a package manager when you've got a package manager?

falcor84• 3 years ago

who needs a package manager when you've got a package manager

That flew over my head; could you explain?

foota• 3 years ago

I think the idea is that being the next C++ means being just as hard to write as C++, and then saying "why do we need the next C++ when we already have C++?", but I'm curious if my interpretation is right :)

binary1323• 3 years ago

Lots of languages have tools which reimplement package management specifically for their libraries, and this pattern has become fairly "normal", but the traditional norm is to access and provide libraries as system packages via the system's package manager. With that in mind, things like Cargo become less magical and more redundant with other tools like Make.

chrismsimpson3• 3 years ago

Suggesting that single threaded concurrency is the right way to do it when building tooling is completely asinine.

IshKebab3• 3 years ago

He isn't suggesting that. He's suggesting it is the right place to start, in the same way that we normally start writing sync code with a single thread.

chrismsimpson3• 3 years ago

Nope. Direct quote:

“..and when you need to utilize multiple CPU cores, you just spawn multiple processes that listen on the same socket. This is a much better way of structuring servers..”

pdimitar26• 3 years ago

Just a personal take: after not coding with Rust for several months, I find it more and more difficult to return to an async code I was writing.

The whole thing just reads... ugly and inconsistent. It needs too much already-accumulated knowledge. As the article correctly points out, you need a bunch of stuff that are seemingly unrelated (and syntactically speaking you would never guess they belong together). And as other commenters pointed out, you need to scan a lot of docs -- many useful Tokio tools are not just not promoted, they are outright difficult to find at all.

Now don't get me wrong, I worked on projects where a single Rust k8s node was ingesting 150k events per second. I have seen and believed and I want to use Rust more. But the async story needs the team's undivided attention for a long time at this point, I feel.

Against my own philosophy and values I find myself attracted to Golang. It has a ton of ugly gotchas and many things are opaque... and I still find that more attractive than Rust. :(

This article is a sad reminder for me -- I am kind of drifting away from Rust. I might return and laugh at myself for this comment several months down the line... but at the moment it seems that my brain prefers stuff that's quicker to grok and experiment with. Not to mention writing demos and prototypes is objectively faster.

If I had executive power in the Rust leadership I'd definitely task them with taking a good hard look at the current state of async and start making backwards-incompatible changes (backed by new major semver versions of course). Much more macros or simply better-reading APIs might be a very good start. Start making and promoting higher-order concurrency and parallelism patterns i.e. the `scoped_pool` thingy for example.

rklaehn19• 3 years ago

I fully agree that the async story needs attention.

Something as drastic as backwards-incompatible changes might not be needed.

But definitely much more documentation about best practices, highlighting the option of local async tasks, making the APIs for that more convenient, and also some low hanging fruits in terms of language syntax.

It seems a bit like while there was a lot of excitement about the async syntax a few years ago, now there is a MVP syntax and everything has just slowed down a lot.

pdimitar26• 3 years ago

now there is a MVP syntax and everything has just slowed down a lot.

Sadly this is spot on. Seems like after async was delivered there was only one big push from tokio 0.1 to 0.2 to 1.0 and that was it, the ecosystem and/or language maintainers seem to have lost interest in making it better to read and use.

insanitybit19• 3 years ago

If you want golang in Rust just use channels and tasks? Or threads?

I don't find async Rust difficult at all, I'm having a hard time really empathizing with this to the extent of needing breaking changes.

To me, async from a lang perspective is virtually done - in 2024 I suspect all of the various impl Trait and async Trait stuff will be done and at that point I don't see anything left.

pdimitar26• 3 years ago

I do that of course, and that's one of the easiest ways to use async Rust. In real projects you need much more however. F.ex. I had to code an example of how to add tasks to an already running pool of tasks and posted my findings here: https://github.com/dimitarvp/rust-async-examples/blob/main/e... (there's #2 as well with some more comments and a different approach).

The fact that I needed to make a GitHub repo and start making show-and-tell demos on how to do various things with async Rust to me is both a red flag and me being diligent BUT it should be more obvious. And promoted in docs.

Rust started suffering from "you got all the nuts and bolts in place, now build your own solution, son" syndrome which I grew to dislike. Too low-level. I wouldn't mind something akin to e.g. Golang's flowmatic library (check the first two examples at the top of the README): https://github.com/carlmjohnson/flowmatic

rklaehn19• 3 years ago

I think this blog post by tomaka gives a good summary of the current issues with async rust. Certainly much more comprehensive than what you can write in a hacker news post:

https://tomaka.medium.com/a-look-back-at-asynchronous-rust-d...

It highlights some of the same problems:

- “Just spawn a new task”

- The Send trait isn’t what it means anymore

jerf5• 3 years ago

I really think in the end, in another decade or two, the community consensus is going to be that async as it is conceived of today is simply a mistake, full stop.

Think about it. Where did it come from in its current incarnation? Node. Why did Node choose it? Did it have a multiplicity of options and carefully choose the best one based on years of experience with each choice? No. Async was "chosen" because it was the only option for the weak runtime that it had to work with. It was the only choice.

But the Node marketing machine rumbled into action and made bold claims about performance and quality and phrased them in universals across all languages rather than merely claiming it was good for Javascript and that propaganda is still running around in people's minds to this day.

I know Rust chose it for performance reasons but I still think in a decade or two community consensus will be either that the 5% performance overhead is worth it for any non-trivial program because async is just too much worse, or someone will come up with an even lower impact version of some other option that will dominate it.

As your message demonstrates, we're still in the phase where you have to pay lip service to the supposed community consensus before saying "but I think it isn't really working out", but as I don't care about the community consensus I don't mind just saying it. Async is a mistake. It's just you manually doing what the compiler ought to be doing for you, and while I'm open to other possibilities in the future, right now threaded code is straight-up a better option on almost every metric, and the metrics where it may have a slight disadvantage, it's only slight and worth it. It's not like you choose threading and you instantly lose 10x performance or something. The first and second derivative opinions about async are clearly negative and I don't mind jumping farther ahead in the process when it comes to my own engineering decisions.

seanhunter2• 3 years ago

Async in its current incarnation did not come from node. Eg Haskell had async/await in 1999. https://softwareengineering.stackexchange.com/questions/3774...

Node wasn't released until 10 years later.

kibwen4• 3 years ago

Where did it come from in its current incarnation? Node.

Rust's async/await is inspired by C# (which was inspired by F#, which was inspired by Haskell).

right now threaded code is straight-up a better option on almost every metric,

Agreed that people are generally too eager to reach for async when they could easily get away with threads, and fortunately Rust makes threads extremely pleasant to work with. The whole point of Rust is that it doesn't force you to use any given paradigm when it comes to concurrency/parallelism, and has support for many approaches.

cmrdporcupine7• 3 years ago

We wrote asynchronous driven network code for decades without syntactic sugar for it, and it was fine. That 5% performance gain can be had without it. Async syntax is a mistake. Not just because of the mess it makes across the program tree, but also because it brings with it a specific notion of how to do asynchronous I/O.

io_uring for example gives an entirely different model, one with some lovely performance benefits. I'm sure there's a way to make it play well with Rust async and tokio, etc. but they're not really the same model on the surface. And so having bolted async into the language syntax itself, and also adding an explicit requirement on concepts like 'executors' into the language, we've created a legacy problem.

IMHO async in Rust was a mistake. A systems programming language should not have mandated a technique like this, and should have stayed agnostic.

insanitybit19• 3 years ago

Saying that Rust chose async because node did is such an absurdly ignorant, incorrect statement.

ahupp• 3 years ago

I used/worked on a similar async model at FB long before I'd ever touched node. Threads are not really solving the same problem as async.

Imagine you are building the FB newsfeed. You have 10 stories and want to fetch them as quickly as possible. The stories are all different types and each refers to other data: profile info, privacy, comments (which depend its own profile fetch), likes, etc.

So you have this tree of dependent data fetches, how do you minimize the time waiting? You could walk the tree node-by-node, but that balloons your wait time. So you kick each bit of work off to a new thread and wait for it to be done. How do you wait for it? Lots of options of course, but they all end up looking like an ad-hoc implementation of async. Which is fine! But if you have a large codebase where a large fraction of code is dealing with this stuff you pretty soon want the runtime to handle this. And if you are mixing code written by others you really want everyone to use the same mechanism.

So the value of async isn't really in the specific implementation choices, it's that it lets you represent this tree in a consistent way. This is less about the number of requests per second, and more about how much code you have to deal with; plenty of places have low RPS but have accumulated lots and lots of code.

Second, threads have their own costs even ignoring performance. Arguably a big reason PHP/Hack worked so well for FB is that each request was single-threaded and started with a clean state; there's whole universes of bugs that are avoided in this model. And node doesn't have threads because JS doesn't have threads, microcontrollers may not have threads, etc.

pdimitar26• 3 years ago

I am not versed in computer science well enough to comment on the model in general but I'll agree that the current way of doing async is pretty sloppy. When you fully invest your project in it and use async libraries it does get better but some compilation errors still sound like dark magic (I'll forever regret not writing down a few of them I encountered last year because the team is receptive to changing rustc error message).

Though I'll also say plain old multithreading and async are serving different niches, in my experience.

jacquesm15• 3 years ago

Any kind of mechanism like that requires a mental model that carries a ton of state because it is no longer immediately visible what the scope of execution is.

Continuations and the way Erlang handles this have far less mental overhead and help keeping the model and the mental representation of that model in sync. Differences between the two is where bugs will hide.

The web isn't asynchronous it's synchronous in almost every aspect the whole reason all this async stuff became so popular (at least, as far as I understand it) is because there is a mismatch between your typical process and having a large number of clients dangling off it. The solution - in my opinion - is to solve this by making it easier to have more processes, which can then block while waiting for the other side, not by adding layer upon layer of complexity to try to stick to the wrong set of tools or to shoehorn a larger problem into a box too small to contain it.

It essentially gets you all of the complexity of interrupt driven code for very little gain and without the memory protection that a process driven approach would have. This whole async solution is to offer an alternative to Node, (even if it originated elsewhere that's clearly the competition for Rust) and should have stayed there, it's an ugly wart on Rust and a distraction to boot.

rklaehn19• 3 years ago

Kind of agree. The erlang runtime just gives you a process abstraction that is extremely cheap. So within a process you can do things logically sequentially without dragging the system as a whole down.

This approach would not have been the right choice for rust itself, since rust as a systems programming language can not have a runtime.

But there is a fun project called lunatic that implements a lightweight runtime in rust https://docs.rs/lunatic-runtime/latest/lunatic_runtime/ and then allows you to write lightweight processes for this runtime also in rust https://docs.rs/lunatic/latest/lunatic/

It seems heavily inspired by the actor model and erlang.

insanitybit19• 3 years ago

Continuations and the way Erlang handles this have far less mental overhead and help keeping the model and the mental representation of that model in sync. Differences between the two is where bugs will hide.

You still have a lot to think about in Erlang. For example, you need an entire supervisory system to handle the fact that an actor can die. You need to handle the fact that an actor A might send a message to actor B and not get a response, or it'll get a response way later. You need a native way to link actors, a native way to name actors, a tracing system, etc. Actors are not something you can simply slap onto a system and suddenly you've got Erlang superpowers.

TBH tokio tasks give you a very similar abstraction without the problems - you can join a task. That's a huge ergonomic improvement because you can isolate/join state at will without having to use a channel.

it's an ugly wart on Rust and a distraction to boot.

It's really not that bad. Like, IMO the major reason to have async is for cancellation. I want to be able to have a network request time out. That is way, way easier to handle in async code than threaded code. The best I can do with a sync system is specify timeouts on an underlying resource like a socket, I can't do anything like "this specific request should have this specific timeout" without async.

pdimitar26• 3 years ago

Oh, hey again. :)

And yes I work mostly with Elixir for years and Erlang's model is just irreplaceable so far. You're quite right that everything is synchronous but all the Erlang "processes" (green threads, fibers or w/e people want to call them) are preemptively and forcibly switched. And that has basically eliminated 95% of all parallel programming problems.

I would kill for Erlang's concurrency / parallel primitives in Rust, especially if they come with their own DSL (likely via Rust proc macros?) and are much shorter and comprehensible than they are right now. :|

It essentially gets you all of the complexity of interrupt driven code for very little gain

Yep, sadly. I've written some fairly impressive commercial Rust code but it added gray hairs and left me wondering whether that was the best way to do it.

pjmlp20• 3 years ago

Rust sweetspot is really for use cases where any kind of automatic memory management is forbidden, either due to real use case requirements (high integrity computing, kernel drivers,...), or due to existing domain culture that frowns upon any other kind of alternatives.

For everything else, there are more productive alternatives, even Go, which after generics is kind of ok.

tjoff8• 3 years ago

Maybe for prototypes or small scripts but disagree otherwise. The promise of automatic memory management is that you don't have to think about memory.

But the second you don't think about memory you are doomed to write bad code anyway. Maybe because of performance but most probably due to architecture. To have a language that forces you to think about memory is not a curse, it is a blessing.

rklaehn19• 3 years ago

Automatic memory management is not exclusively positive, even if you don't care about performance.

If you lose deterministic destructors and therefore the possibility of RAII, for me it is a loss.

I worked with lots of garbage collected languages (Java, C#, Scala) before rust, and I don't miss garbage collection at all...

nlitened• 3 years ago

Honestly, is 150k events per second even a meaningful number to talk about in performance context? I've got Clojure code ingesting 150k messages per second, processing, then outputting 30k msgs per second sitting at 47% CPU on a cheapo 70 $/mo VPS. Straightforward unoptimized C/C++ achieves millions of ZeroMQ messages processed per second in a single thread.

Why use async or even Rust at all at such small loads?

pdimitar26• 3 years ago

I knew somebody will call me out since I didn't give details. :D

It was a 1-2 vCPU k8s pod which is still pretty impressive IMO. On my 20 CPU threads workstation I can easily achieve multi-million events per second. Even one of my Linux laptops can go at around 1 million but its ETH interface started overheating and couldn't sustain it longer.

Why use async or even Rust at all at such small loads?

That's the better question, yeah. We had 200+ k8s pods and had to ingest a lot of data is the simple answer. Using anything except C++ or Rust would have made our cloud bill much bigger.

Though I have to admit that nowadays I would be very strongly tempted to try with Golang or even OCaml's new multithreaded runtime.

lenkite2• 3 years ago

No, it is completely fair to say that reading and comprehending Golang concurrent code is far-far easier than reading Rust Async code. With Virtual Threads and Structured Concurrency, even concurrent Java code is easy to read nowadays.

Rust Async is far more difficult to understand than Rust Sync. You can't just dive in. You need to have coffee and truly focus your mind like you are solving a Math Olympiad problem. I find it difficult to believe that a community language came to this ridiculous level of complexity.

pdimitar26• 3 years ago

Yikes. You summarized it much better than I could, kudos.

vacuity6• 3 years ago

The constraints of performance and whatnot limited the options for Rust's async model. It's not bad given the constraints, but it's not great, and how developers use async Rust vs. Go matters even knowing this.

insanitybit19• 3 years ago

I'm sympathetic to this point but I think that:

a) Saying Node + Deno are good is a stretch. Node has horrible performance, even for simple routing. And I'll source that[0].

b) Saying that adding `Send + Sync + 'static` bounds is a serious burden is, to me, overstating things.

the far better model for writing performant servers.

It's completely workload dependent. For a chat server it's almost definitely not going to be more performant and you may end up with worse latency.

it only costs you friction everywhere else in your entire codebase, and quite often performance as well.

I am unconvinced tbh. I do not believe that adding Send + Sync + 'static bounds is onerous, I do not believe satisfying those bounds is hard (it's almost always just a matter of moving the value), and I do not believe that the vast majority of programs benefit from TPC architecture.

I recognize that there is a problem here - that we are optimizing for one runtime at the expense of others - but I am not convinced at this point that the problem matters.

[0] https://www.techempower.com/benchmarks/#section=data-r21&tes...

captainmuon2• 3 years ago

Wait, async is multithreaded by default in Rust? For me the whole point of using async in JavaScript or Python (originally with Twisted's @inlineCallbacks) was to get concurrency without threads.

Imagine writing code for a computer game bot: move left, wait for enemy, attack enemy... You normally can't write it like this because it would block the rest of your program. Async allows you to go from "program sequential" to "bot sequential" for lack of better terms. If you are IO bound there is no need for threads. I often like to use one network thread and one GUI thread to keep things separate, and to prevent the occasional blocking in one to cause latency in the other. You just need to have a method to post calls to the other event loop. Works well in Python, as well with Qt-based apps.

C# on the other hand made the same "mistake" of being very general. I think you can even have a coroutine suspend on one thread and wake on another. It looks like you switch threads in the middle of a function.

I guess that is useful when you want to write performant multithreaded servers. I just want to write easy sequential code without worrying about locks or state machines.

maeln• 3 years ago

Wait, async is multithreaded by default in Rust? For me the whole point of using async in JavaScript or Python (originally with Twisted's @inlineCallbacks) was to get concurrency without threads.

Async in rust is nothing by default because it doesn't have a runtime. You need to use a runtime/executor for async to work, the most popular being Tokio, which can be configured to be single-threaded or multi-threaded.

EDIT: But in a way since Rust make 0 assumption about how async function are going to be executed, it does design around the fact that they might be multi-threaded.

kibwen4• 3 years ago

Wait, async is multithreaded by default in Rust?

No, whether or not async is multithreaded depends on whether or not your executor is multithreaded. Rust doesn't ship an executor by default, you need to bring your own. Tokio, the most popular general-purpose executor, is multithreaded by default, and can be configured to be single-threaded.

duped7• 3 years ago

The point being made isn't whether or not the futures are evaluated on one thread or many, but that by default, library authors assume that something may be evaluated on multiple threads which imposes some constraints on both the argument and return types from functions.

So for example, the tokio executor could run single threaded or on a thread pool. However, tokio::task::spawn takes a future as an argument that may be evaluated on a different thread, so any caller of this API and the designers of the library behind it need to guarantee the argument is thread safe.

That constraint is reflected through the type system by constraining the argument type of tokio::task::spawn<T>(fut: Future<Output = T> + impl Send + 'static) which means "fut is a type that implements Future with the output of type T and is Send (safe to send to a different thread) and does not have any lifetimes that outlive the 'static lifetime (contains no references to non-static data)".

captainmuon2• 3 years ago

That's just what I was trying to say. If your language potentially allows futures to resume on different threads, you pay a complexity cost. I'm not too familiar with Rust, but I think the cost is especially high here because Rust is so concerned with correctness (there is no Send trait in C++ for example).

And if you are using async only for nicer control flow, then there is no need for multithreaded executors. Just coroutines that live on a single thread and get orchestrated by an event loop (trampoline/reactor/executor).

rich_sasha• 3 years ago

Moaning aside, what is (if any?) the direct equivalent to single-threaded asyncio, like Python or node.js, in Rust?

The thing I enjoy about async in Python is that it's very easy to write "thread"-safe code - you know exactly where you might give up execution context, and 90% of the time you have no need for mutexes and locks. But as this article complains, in Rust, it seems to be sync or multi-threaded.

zokier5• 3 years ago

The article mentions smols LocalExecutor and tokios LocalSet, both which are single-threaded

packetlost4• 3 years ago

I'm still not sure what async (cooperative multitasking) gives over green threads (preemptive userspace multitasking)

surajrmal3• 3 years ago

Lower overhead. It's the same reason you would need to avoid a GC. It may not be for everyone, but that isn't a rust goal necessarily.

packetlost4• 3 years ago

I'm aware that Rust's threading has higher overhead because they're system threads, but what about green/user threads? Is there intrinsic overhead to userspace threads that async doesn't have? I think every "task"/thread needs it's own callstack space, and you're paying scheduling overhead no matter what. Is there literature on the topic?

yencabulator6• 3 years ago

It's not that simple.

The article is specifically complaining about how the design of Rust's async ecosystem forces one to use wasteful Arc<Mutex<T>> etc in places where the data has no reason to move across threads.

Meanwhile, a greenthread system can happily use local state without atomics.

I personally want Tokio and Glommio to have a baby that inherits good parts of both parents.

gwbas1c• 3 years ago

When I was between jobs, I decided to learn rust by writing some async code. I really got stung and spent days doing things that would take minutes in C# or Node.

Part of the problem is that, in Rust, stack memory is easier to use than heap memory. When writing traditional threaded code, this isn't much of an issue because naturally most of our code is working with values on the stack.

BUT: When we look more closely in how async works in C# or Javascript, the compiler, under the hood, breaks up an async method into multiple methods and puts values that appear to be on the stack onto the heap. (Of course, I'm over simplifying.) It just works, and it just works well.

But, in Rust, making something async implicitly moves what appears to be on the stack onto the heap. It can quickly become hard to reason about.

I wish I knew about techniques that this article describes. Maybe it would make my code easier? In my current hobby projects, I'm doing traditional blocking IO because it's not "worth it" to write async. (In comparison, in Node, async code helps avoid nesting callbacks within callbacks, and in C#, doing IO in async is a best practice.)

pie_flavor• 3 years ago

The thing the article calls bad is not the thing the examples illustrate. The set-up is supposed to be that multithreading is a pain, but none of the examples actually do that. Take the initial list. "These are not async primitives, these are multi-threading synchronization primitives." Yeah, but they're multi-threaded forms of stuff you still need. If you needed RwLock in multi-threaded mode, you need RefCell in single-threaded mode, and the API is virtually identical. If your state needed to be in an Arc, it'll need to be in an Rc. And if you needed sync::mpsc, you'll instead need... sync::mpsc.

A Send bound is no great burden. The only type that you will regularly interact with that is not Send is the lock type from a Mutex or RwLock, which is good because if you hold it across a long await you can slow down your app, a bug prevention mechanism that does not exist in straight multithreading. The only point that actually illustrates a threading-caused problem is the thing about multithreading sockets, which it admits is almost imperceptible, and which you can also solve by not doing that.

Almost everything the author identifies as a parallelism problem is a 'static problem. Tokio is missing a scoped spawn like std has, and if it gained one then the much described multithreading woes would reduce to basically nothing.

Animats• 3 years ago

It's a frustrating area. As I've mentioned before, I'm writing a high-performance metaverse client in Rust, something which has many of the problems of both a web browser and a MMO. If you want to have a good looking metaverse, it takes a gamer PC multiple CPUs and a good GPU to deal with the content flood. (This is why Meta's Horizon looks so bad.) Now you have to use all that hardware effectively.

So what I'm writing uses threads. About 20 of them. They're doing different things at different priorities towards a coordinated goal. This is different from the two usual use cases - multiple servers running in the same address space, and parallel computation of array-type data.

Concurrency problems so far:

- Single-thread async is simple. Multi-thread async is complicated. Multi-thread async plus other threads not managed by the async system isn't used enough to be well supported.

- Rust is good at preventing race conditions, but it doesn't yet have a static deadlock analyzer. It needs one.

- Different threads at different priorities do work in both Linux and Windows, but not all that well. With enough low-priority compute-bound threads to keep all CPUs busy, high-priority threads do not get serviced in a timely manner. I was spoiled by previous work on QNX, which, being a true real-time operating system, takes thread priorities very seriously. On QNX, compute-bound background work has almost no effect on the high-priority stuff. Linux just doesn't work well at 100% CPU utilization. Unblocking a lock does not wake up a higher priority waiting thread immediately. This can delay high-priority threads unnecessarily.

- The WGPU crowd has spent a year getting their locking sorted out so that you can load content into GPU memory while the GPU is rendering something else. It's a standard feature of Vulkan graphics that you can do this, but it has to be supported at all levels above Vulkan too. For me, that's WGPU and Rend3. That stack is close enough to ready to test, but not ready for prime time yet.

- There's no way to cancel a pending HTTP request made with "ureq". "reqwest" supports that, but you have to bring in all the async and Tokio stuff, which means you now have multi-thread async plus other threads. This is only a problem for what I'm doing when the user closes the window, and the program needs to exit quickly and cleanly. I'm getting a 5-10 second stall at exit because of this.

- Crossbeam-channel is not "fair"; it's possible to starve out some requests. Parking-lot is fair, but doesn't have thread poisoning, which means that clean shutdowns after a panic are hard.

- Running under Wine with 100% CPU utilization with multiple threads results in futex congestion in Wine's memory allocation library, and performance drops by over 99%, with all CPUs stuck in spinlocks. The program is still running correctly, but at about 0.5 frames per second instead of 60 FPS. Bug reported and recognized by the Wine crew, but it's hard to fix. I can make this happen under gdb running my own code and see all those threads in the spinlocks, so I was able to file a good bug report. But I haven't generated a simple test case. It's a Wine-only problem; doesn't affect Microsoft Windows.

So that's life in a heavily threaded world.

Individual CPUs have not become much faster in over a decade. Everybody has been stuck at 3-4 GHz for a long time now. CPUs with many cores are widely available in everything from phones to game consoles. To use modern hardware effectively, you need threading.

iopq2• 3 years ago

AMD's new processors easily double FPS in modern games

I bought a 3600 in 2019, but a 7800x3D would blow it out of the water after just 4 years due to more cache, hitting 5Ghz, faster DDR5 RAM

https://youtu.be/B31PwSpClk8?t=800

shadow of the tomb raider, 152 FPS to 382 FPS

some games wouldn't double the FPS, but I'm really just looking to hit 280 FPS on my 280Hz monitor

dathinab2• 3 years ago

why multi-threaded task executors should be the default

many many reasons and it's subtile and complicated

For one ironically it's just way easier to use, especially for less experienced programmers. In a task system like that it's just way easier to accidentally cause major issues when it's many tasks on one thread compared to multiple threads (at least with the guard rails rust provides for threading safety). On the other hand the performance overhead of by-default multi-threaded is just fine for a ton of use-cases to a point you could argue worrying about it is premature optimization.

Through it's important to state that a lot of this comes from the ecosystem around rust and not rust itself, as stuff like `LocalSet` shows you can have a non-multi threaded runtime and there is no reasons all the libraries you might use couldn't provide non multi thread safe versions. Some do. Just many decided that avoiding the performance overhead of being thread save isn't worth the maintenance overhead and additional foot guns it can provide.

Now naturally you can say "but node/deno/etc." but they are a completely different beast then "just" being not multi threaded by default. Like e.g. they don't have multi threaded code at all. Just single threaded code communicating through serialized messages (kinda). They also e.g. handle all I/O completely separated from you application code and don't have any non serialized communication between threads etc. etc.

Interestingly if you look at the design choices of the I/O Event loop (reactor) of tokio there are some conceptual similarities. Also there AFIK there is a company which is building something similar to the node model for rust using WASM.

I mean in the end the node-style approach is grate for building servers, but rust isn't just for building servers but much more general with much more low level use cases.

Now the main point where rust could improve quite a bit is to make it much easier to write a library which work very efficient with both cases. Code which crosses thread boundaries and code which doesn't. Currently you are often split between either implementing it twice, doing terrible unusable generics tricks or using tons of `cfg` (probably generated using macros/annotations) and hope no dependency accidentally imports the multi-thread feature when you don't need it. Non of this is really viable but it's a surprisingly hard problem. Currently the best idea I can come up for it is generic modules which make the "terrible to use generics tricks" usable, but it's probably not enough by a long stretch. (Even if a solution is found it might not work for Waker, even if it does you still might want sync/send Wakers in some cases.)

dathinab2• 3 years ago

Rust asyncs design was probably the mostly correct decision for what rust is: A general system programming language which you can use in most situations people today use C,C++ and more.

If rust would only be targeting web server programming the right decision might have been no async and green threads, it's just much easier to use.

But that rust would likely never have succeeded as most of it's initial success cases where for use-cases where you wouldn't want green threads.

Nicely we might still get no async and green threads: In form of run times which run WASM compiled rust code in a node like fashion. Probably in combination with some serverless/edge-compute providers which hopefully will be nice to use.

paletteOvO• 3 years ago

async/await is ugly and hard to use and understand for me. It is pretty reasonable that rust chose it because it is a zero-cost abstraction. But i just don't like it.

Local async executors and why they should be the default

Related Stories