Narrowing the notion of a runtime in Rust

https://github.com/rust-lang/rust/commit/0efafac398ff7f28c5f0fe756c15b9008b3e0534

by heinrich5991 • 12 years ago

208 102 12 years ago

btown• 12 years ago

Is this going to be part of the 1.0 release schedule, then? It seems ridiculously breaking when they're only 20 days from stable!

http://blog.rust-lang.org/2014/12/12/1.0-Timeline.html

db48x• 12 years ago

20 days away from the 1.0.0-alpha; it'll be at least 12 more weeks before the actual 1.0.0 release.

bluefinity• 12 years ago

It's 20 days until the 1.0 alpha -- from the timeline in that blog post, stable would be March 30 at the earliest.

kibwen9• 12 years ago

Don't expect the January 9th alpha to signal the end of breaking changes. It merely represents a significant raising of the bar for breaking changes to be accepted. The subsequent betas will raise the bar even further.

In fact, as we approach the alpha, I expect several frantic last-minute breaking changes as people attempt to beat the buzzer.

blt2• 12 years ago

Why did you decide to use hard deadline project management? It seems like Rust's ideals of correctness and fixing fundamental C/C++ flaws are more aligned with a "when it's done" release. On the other hand, the language needs a stable release to pick up many users. I'm guessing the latter was your motivator. And knowing that real world users will likely find problems with your design no matter how long you spent perfecting it. Just curious what went into the decision.

buster• 12 years ago

So.. you want that change AFTER going stable or to not happen at all?

Ruud-v-A• 12 years ago

The merge may have occurred just yet, but the actual pull request and first draft of this change are several weeks old.

alexanderblom• 12 years ago

Link to the PR: https://github.com/rust-lang/rust/pull/19654

sciurus• 12 years ago

and to the RFC: https://github.com/rust-lang/rfcs/blob/master/text/0230-remo...

jephir• 12 years ago

Does removing the runtime have any effect on the current effort to get Rust working on Emscripten?

mrec• 12 years ago

I was wondering that myself. I remember it being given as the main blocker when someone asked about Emscripten, but that was over a year ago and a lot has changed since.

kibwen9• 12 years ago

Very simple and stripped-down Rust programs have already been demonstrated to work via Emscripten. I expect the work here would make it a little bit easier, but I don't know of anyone actively working on supporting this use case.

Last I heard, more advanced support hinged on Emscripten upgrading their version of LLVM, which was much more ancient than the bleeding-edge version that Rust uses.

NickPollard• 12 years ago

This looks to me to be an important and very promising change. I don't know how much they've had to compromise, but making this easily callable in from C (and hence, easily callable from other languages with C interop) makes it simple to embed rust in other languages and applications.

For example, I'd been looking at doing Rust development for Android; currently native (i.e. C/C++ apps) on Android get run through a Java wrapper; starting a Rust process with the Rust runtime active is incredibly complex. With this change, it should be trivial to have a basic Android activity that just calls a Rust main() and have everything go from there.

patal• 12 years ago

Does anyone know if this makes using Rust code on an Arduino easier? There's [https://github.com/jensnockert/dueboot] which only works on the Due, not the Uno.

cmrx64• 12 years ago

Still limited by lack of an LLVM backend for AVR, doesn't make it any easier or harder.

lambda• 12 years ago

I think the main blocking part there is getting LLVM working on AVR. I believe there is a fork of LLVM that does (https://github.com/sushihangover/llvm-avr), but you would have to merge that fork with the LLVM that Rust uses in order to run Rust on it, and it looks like the fork is a little out of date so it may take some work.

This particular likely doesn't affect running Rust on an Arduino at all, as what it is doing is removing the runtime for std, which contains many of the OS abstractions. You don't have an OS on an Arduino however, so std isn't particularly helpful there. Several months ago they already did what was necessary for that, factoring out much of the completely OS independent functionality into a "core" library (http://doc.rust-lang.org/core/), so you can opt out of using std and use "core" instead, plus add a little bit of platform specific glue, to run Rust on bare metal.

pjmlp• 12 years ago

Not what you want, but for Arduino safe systems programing one can in the mean time use

Ada, http://sourceforge.net/p/avr-ada/wiki/Home/

Pascal, http://www.mikroe.com/mikropascal/avr/

errordeveloper3• 12 years ago

The dueboot project is rather dead, checkout zinc.rs :)

IshKebab• 12 years ago

ARM Cortex chips are better than Atmel in virtually every way. The only sometimes-disadvantage is that they are 3.3V rather than 5V (though that is often an advantage these days).

So I doubt many people will be motivated to work on this.

steveklabnik2• 12 years ago

While it's true that this is a pretty big win, both were already supported. The second production deployment of Rust is a Ruby gem, written in Rust, and we test each commit against Android.

xfalcox• 12 years ago

I'm not a low level guy, but this means we can have rust libs callable from ruby gems through C interop?

Isn't this fucking awesome?

rubiquity2• 12 years ago

You can already do this. I think this will just help make it easier and even more lightweight. I'd love to know the exact details, though.

kibwen9• 12 years ago

Not only is this already possible, but the largest production deployment of Rust does exactly this (www.skylight.io).

Here's a dummy repo from steveklabnik to demonstrate the process: https://github.com/steveklabnik/rust_example (though the change in the OP is so now that I doubt this repo has yet been updated)

steveklabnik2• 12 years ago

I compiled a Rust yesterday, ran `make`, and it worked, sooo...

imron• 12 years ago

This is going to make rust so much easier for me to use in serious projects. I have a ton of stuff in c++ that I can't realistically just stop using. I'd love to be able to start introducing rust into those code bases however and this will make things much less hassle.

cosarara97• 12 years ago

I'm more surprised to find Lovecraft quotes in Rust's source code:

https://github.com/rust-lang/rust/blob/6bdce25e155d846bb9252...

kibwen9• 12 years ago

And in your Rust binaries, though this is a bug long since acknowledged :) https://github.com/rust-lang/rust/issues/13871

(And one of those isn't a Lovecraft quote, it's a Majora's Mask quote.)

ikawe• 12 years ago

Ok, I don't know rust at all, but often when I'm trying to understand a change, I look at how it affects the test cases.

Seeing the diff in the test cases should show how the API has changed, but I'm having a hard time reconciling the description of the changes with how the tests have changed.

There are several occurrences of changes like this:

    - spawn(move|| {
    + Thread::spawn(move|| {
         tx.send(i);
    - })
    + }).detach()

Thread methods have been namespaced under "Thread::" and now a "detach" method is appended to each call to spawn.

Can anyone shed some light?

maninalift• 12 years ago

The description notes the replacement of many uses of "tasks" with native OS threads. Tasks may map to threads or may be managed as green threads.

Threads unlike tasks must be detached if they are too be allowed to outlive their parent.

kibwen9• 12 years ago

In ancient Rust, the standard library attempted to provide an abstraction over the underlying threading model. The purpose of this abstraction, called a "task", was to allow libraries to make use of the standard concurrency constructs while allowing the consumers of those libraries to determine whether or not to use 1:1 threading (the typical platform thread model) or M:N threading (green threads), and it would all Just Work with no effort required.

This was quite an exciting idea at the time. However, in practice, it just didn't pan out: the compromises required to paper over the differences between 1:1 threading and M:N threading managed to almost entirely obliterate the respective advantages of both models.

As a result, Rust is in the process of removing the "tasks" abstraction and replacing it with a bog-standard 1:1 threading model (though it still benefits from all the typical memory-safe and concurrency-safe abstractions that Rust otherwise provides). All users of concurrency in the stdlib will now be using native threads, and it will be left to outside implementors to experiment with green threading libraries (see https://github.com/carllerche/mio for one example of such).

One of the consequences of this change is that the `std::task` module has been replaced with `std::thread`. Formerly, every Rust program imported the function `std::task::spawn` to use for spawning tasks. With the removal of `std::task`, the newly analogous function from `std::thread` has not been added to the prelude, and is therefore not imported by default in every Rust program. It could be added if such a thing were desired (you can see remarks to this effect in the comments on the PR itself (https://github.com/rust-lang/rust/pull/19654#issuecomment-66... and https://github.com/rust-lang/rust/pull/19654#issuecomment-66... )).

mietek4• 12 years ago

What a disappointment.

MichaelGG4• 12 years ago

So now, will an "unhandled" panic on a thread abort the process? Or just get silently ignored until another thread stumbles upon a call that's poisoned?

qznc• 12 years ago

This was quite an exciting idea at the time. However, in practice, it just didn't pan out: the compromises required to paper over the differences between 1:1 threading and M:N threading managed to almost entirely obliterate the respective advantages of both models.

I would love to read a longer article about this. M:N has quite an allure, but it never seems to reach mainstream. Erlang and Go provide it and they seem to be happy. Both languages do not focus on performance, though.

moca• 12 years ago

Java tried green thread since day one in 1995, and it never worked out well and was dropped long time ago (around 2000). Go did a good job with its gorountine at certain cost, for example embed Go library is much harder than C library, so Go application is typically a standalone static binary.

It is really great to see Rust drops the mixture of green and native threads, and reduce the runtime to minimum. It will make Rust stand out in many environment that no other languages would work well (besides C).

MichaelGG4• 12 years ago

Now to remove unwinding and make it optional, or make fully recoverable exceptions... This seems like the biggest controversy in Rust, and it sounds like the plan is to just make unwinding optional, though last I read, that means two incompatible stdlibs and s need to compile everything in two different modes. Still it'd be really nice to not have unwinding when your program doesn't need it, but also be able to make apps that benefit from exceptions, like web servers.

fanf2• 12 years ago

Isn't unwinding a standard part of C/C++ ABIs these days? I would have thought that disabling it would cause more compatibility problems than supporting it.

Animats• 12 years ago

They're getting rid of trying to do their own CPU dispatching. (Some people call this "lightweight threads" or "green threads"; Go's "goroutines" work that way.) That simplifies things considerably.

There's a compile-time optimization which does many of the things people want to do with such threads. If you have two tasks communicating over a channel or queue, only one task writes to the queue, only one task reads from the queue, and the maximum length of the queue is 1, the code can be compiled as a coroutine. Go may get that optimization at some point; it's been discussed.

(Things that have been learned about Go goroutines: 1) the above case is moderately common, 2) for "select", the most common case is N=1 alternatives, followed by N=2. N>2 is rare. Go has special case code for N=1, and the case for N>1 is very general and does a lot of dynamic allocation. A special case for N=2, which in practice is usually read-or-timeout/abort, would be useful.)

dbaupp7• 12 years ago

Support for green threads (and hence any manual CPU dispatching) were actually removed a month ago[18967] along with much of the generic infrastructure for mediating between 1:1 and M:N threads, the patch linked here just completes the removal of that infrastructure and switches things to be much closer to the raw OS in implementation and semantics.

[18967] https://github.com/rust-lang/rust/pull/18967

Juan_Shapiro2• 12 years ago

What impact will big changes like these have on an eventual stable release of Rust? My team has been reading about Rust for a long time now, and we've tried it out, but we just can't deal with the constant change. We need a language that offers what Rust can offer, but we also need our code to still work without changes a few months after we write it. Earlier this year we were hearing that a stable release of Rust would be out before the end of 2014. But that clearly isn't going to happen. The latest I read was that it's now supposed to be ready sometime in March 2015 at the earliest. I understand that software schedules slip, and that it's important to get things done right the first time around whenever possible, but getting out a stable release of Rust just seems to be dragging on and on! I really, really hope that Rust 1 is available by this upcoming March, but if history is any indication, that may be unlikely to happen. Big changes like these don't bring me much comfort at all.

saosebastiao• 12 years ago

Is it possible to have a threading model without an OS kernel? In other words, is it possible to package the rust runtime with an embedded execatable and run threaded software without the OS? Or does threading inherently require the OS to do the thread scheduling, etc.?

dexen• 12 years ago

Plan 9's libthread [1] implements cooperative threading entirely in userspace library. This is apart of Plan 9's fork() [2] syscall which allows both forking processes and creating kernel-level threads.

Thanks to libthread being orthogonal to processes, it turned out to be easily portable.

[1] http://man.cat-v.org/plan_9/2/thread

[2] http://man.cat-v.org/plan_9/2/fork -- this is what linux' clone(3) syscall was modeled after.

andrewchambers• 12 years ago

It seems to me like the designers really couldn't make up their minds.

Do you want GC or not? Do you want lightweight threads or not? What level of runtime requirements do you want?

Its finally converging on something sane, but its taken a while.

mattdw• 12 years ago

Or, to put it another way, the design process happened in the open, and you got to see some perfectly normal things that just wouldn't usually be visible to you. (And even have some input into them, if you were so inclined.) I don't really think that deserves criticism.

jeffreyrogers• 12 years ago

I've thought similar things when I've looked at Rust in the past. It seems to me that designers really wanted to make it nice from a PL theory perspective, (hence the focus on functional programming and a strong type system). I'm more partial to the Go style of language design, which focused on making it easy to do the things the designers did in practice.

I don't want to make it seem like I'm coming down too hard on Rust however, and I'm excited to see how everything works out once version 1.0 is released.

jeffdavis• 12 years ago

For the time I've been interested in rust, it's been quite focused on a very practical problem: safe, without requiring a GC.

Dewie• 12 years ago

I've thought similar things when I've looked at Rust in the past. It seems to me that designers really wanted to make it nice from a PL theory perspective, (hence the focus on functional programming and a strong type system).

Do you have any alternatives to implementing a strong type system in order to give certain well-defined safety guarantees for anyone using the language? Any other alternative theories or implementation strategies that would be better?

I'm more partial to the Go style of language design, which focused on making it easy to do the things the designers did in practice.

The designers/implementers wrote the first non-bootstrapped compiler in OCaml, a functional language. How's that for "things the designers did in practice"?

As far as doing what they were doing in systems programming in practice: the point of the project is to actually get away from their then heavy use of unsafe languages like C++. But then they took the concepts of smart pointers, ownership etc. (presumably inspired by best practices in C/++) and tried to make it bulletproof to use.

iopq• 12 years ago

That's what always happens. You have a few goals that you won't compromise on (memory safety and same performance as C) and then you try to make decisions about things you CAN compromise on.

Havvy• 12 years ago

To be fair, GC was always a big TODO item that became less and less important to implement until there were no go reasons to implement it in the language core (as you can implement it on top of the language), and removed from that list.

The other two are really part of the same coin. Having green threads involves having a runtime that has to be started. Deciding to remove the runtime means removing the green threads.

Narrowing the notion of a runtime in Rust

Related Stories