The state of the Rust Runtime

Tasks are Rust's unit of concurrency. There are two models for scheduling tasks. 1:1 means that each task runs in it's own operating system thread, and the OS is responsible for scheduling. M:N means that M tasks run on N threads Typically, N is roughly one per core, and M is greater than N. The runtime is responsible for assigning tasks to threads and scheduling multiple tasks on a thread. "Green threads" is a term from the Java world, and it means M:N, often with N=1, regardless of how many cores are available.

The motivation for M:N is that OS threads tend to be fairly heavy in terms of memory usage, scheduling overhead etc. For a language that intended to enable highly concurrent programs, it's theoretically more efficient to use one OS thread per CPU to get maximum parallelism and layer the units of concurrency (tasks in this case), on top of that.

The thing is, doing M:N well enough to actually achieve the theoretical gains is difficult, and there are many cases of projects that started with M:N and eventually ditched it and moved to 1:1.

(Edit for clarity on green threads.)

dded2• 13 years ago

Does M:N imply shared memory for all tasks on a thread?

bad_user2• 13 years ago

The problem with M:N is that the kernel has better info on doing efficient scheduling. To make M:N work with preemptive scheduling, the process' runtime would have to communicate with the kernel and agree with it on the best scheduling strategy. And currently, this is a recipe for disaster.

This is why M:N scheduling is in practice cooperative and Rust currently does this too. How it works - basically, when pseudo-blocking I/O calls are made, the active task has to yield (by itself) the execution to another task waiting in line, while it "blocks" for the result. It's very similar to what happens in Scala when dealing with Futures/scala-async, or what happens in C# with "async", or what happens in Clojure with core.async.

Above I've said "pseudo-blocking" for a reason, because it depends on language features to execute non-blocking I/O operations as if they are blocking. The problem with this is that truly blocking operations will block the OS thread for real.

Also, this architecture is only a win if you're in a context in which your tasks are mostly CPU-bound, or if you don't have enough threads to block (e.g. like in the context of servers responding to web requests, in which the upper limit is unbounded). Blocking I/O yields better throughput than asynchronous I/O. In terms of throughput the JVM's blocking I/O primitives are beating any M:N architecture, like Golang, because yes, epoll comes at a cost. Heck, even asynchronous I/O on top of the JVM has better throughput, again because of 1:1 multi-threading.

it's theoretically more efficient to use one OS thread per CPU

This is not true.

In case you're not talking about pure CPU-bound tasks, that CPU core will be left at less than 80% usage and you can't really have pure CPU-bound tasks (even CPU-bound tasks have to wait for I/O, even if asynchronously, which at the very least involves interactions with the kernel). This depends of course, but if you've got a CPU core that's used at less than full capacity, then there's room for another thread or two on that core. Which is why the optimal default configuration for CPU-bound tasks are thread-pools with a number of threads that's 2 times the number of CPU cores, which is a reasonable default left to be improved by real profiling.

Also, blocked threads DO NOT consume CPU resources. The real problem is when those threads wake up, participating in context switching. Which is why, if you've got threads blocking for I/O, it's not a problem if you have 100-200 threads that mostly sleep, waiting on some resource to be available.

To give you an example, on top of the JVM it's not a problem if JDBC is based on blocking I/O. You can have like a thread-pool that waits for SQL queries to execute and when the result comes in, you pass the results to another thread-pool meant for CPU-bound tasks. That's why well-grown JVM application are using at least 2 thread-pools to execute tasks - one for CPU-bound tasks with a size directly proportional to the number of CPU cores and one meant for I/O bound tasks, which is unrelated to the number of CPU cores. As an aside, doing this is also good for avoiding dead-locks from combinations of CPU-bound and I/O bound tasks interacting with each other on the same thread-pool.

BTW, there isn't a single platform that went with M:N threading without regrets. That's because you can easily build M:N cooperative multi-threading on top of 1:1 preemtive multi-threading, but this doesn't work the other way around.

cpeterso2• 13 years ago

Alex Chricton presented these slides at a Rust meetup in December 2013. A video of the presentations is available:

https://air.mozilla.org/rust-meetup-december-2013/

vanderZwan• 13 years ago

Is there a way to download this? It won't buffer properly for me for some reason.

(EDIT: Also, I don't know who made that video interface, but a "HD" button where clicking does not make it obvious whether or not you set or unset the HD option and have to wait for the video to load or not is pretty terrible)

cpeterso2• 13 years ago

I like to watch the Air Mozilla videos offline, so open Firefox's Web Console, play the video, then copy the webm URL from the console log. :)

For this particular video:

  wget https://d3fenhwk93s16g.cloudfront.net/9r2j9t/webm.webm

moomin• 13 years ago

I'm curious as to what the perf figures would be for windows. My gut tells me threads are relatively more expensive and async relatively cheaper than on Linux.

The state of the Rust Runtime

Related Stories