Rust: Enable WebAssembly backend by default

https://www.hellorust.com/news/native-wasm-target.html

by a_humean • 9 years ago

473 228 9 years ago

Maybe I just can't find it, but the largest blocker for using Rust on the web for me is a lack of a exposure/binding system (like embind).

I'm not particularly interested in writing entire webapps in Rust but I do see it being useful for smaller self-contained modules inside a larger JS app. Not having a robust binding API makes that difficult.

EDIT: But unrelated criticisms aside, congrats on the progress! I'm excited to see more first class support of WASM from languages other than C/C++.

arbie• 9 years ago

I am excited for WASM for the same reason; not for wholesale migration, but targeted optimization. Hope documentation for this (likely very popular use-case) improves as WASM matures.

cormacrelf2• 9 years ago

Can anyone describe the (overhead) cost of calling into WASM code in the implementations so far? A lot of optimize-able areas are things like RxJS or Promise implementations (eg Bluebird), or generally libraries making the stack trace like [userspace code] -> [library code, approx 20 stack frames] -> [more userspace]. Should we think of the WASM-JS bridge cost like system calls, or like inline assembly?

themihai4• 9 years ago

I think the real issue is the lack of DOM access rather than binding to JS.

tyingq3• 9 years ago

Supposedly they are adding things like polymorphic inline cache, garbage collection, threads, etc, that would make it more like a VM. Perhaps enough that Python, PHP, Ruby, or similar languages could run in WASM without downloading the whole runtime. That plus native DOM access might make the front end web as diverse as the back end web.

I'm curious whether the fragmentation that would bring on is a net add, or a net drain on the web as a whole.

ralfn2• 9 years ago

Isn't the point of the whole architecture that from a security perspective all IO will always be through JavaScript.

Replicating the attack surface of the browser in JavaScript land in WebAssembly land goes against the whole design of it.

It's for the community to make nice libraries that expose the browser features to WebAssembly land in JavaScript. It's also the right place to deal with cross browser differences.

kenOfYugen• 9 years ago

What do you mean by 'first class support' of WASM in C/C++? Is it somehow possible to produce wasm binaries without utilizing emscripten?

RX14• 9 years ago

Yes, llvm has built in wasm support for a while now.

steveklabnik26• 9 years ago

I agree that the best use-case here isn't full apps in Rust, but writing high-performance modules.

The webpack team is working on stuff so you can just drop a rust file into your proejct, and it will do everything needed for super easy integration.

https://crates.io/crates/stdweb sort of gives you the bindings you want, by the way.

bluejekyll7• 9 years ago

This makes me so excited! Great job to Alex and the entire team getting this done.

I know Rust is not the most perfect language for some. For me, it’s really exciting that there is finally a high level language which can scale for every potential use case.

Rust is great for kernels and the embedded space; Excels in the systems space; Is gaining traction in the native application space; Has great tooling for web backends; and, now, with WASM, it’s capable of targeting the web (Yes, without DOM access this is limited, but my understanding is that is coming at some point in the future).

Are there any other runtime free, memory safe languages that can target all of those use cases?

tyingq3• 9 years ago

"Rust is great for kernels and the embedded space"

I'm no expert, but from a layperson's perspective, the rust usage in these areas depends on a lot of "unsafe" blocks, fighting the borrow checker, etc.

Does that ratchet it down from "great" for this space to something like "ok" or "usable"?

Or is this just the best we can do in terms of a memory safe language for kernel and/or bare metal embedded?

geofft2• 9 years ago

The point of unsafe blocks is to contain unsafety. In a language like C, the natural pattern is to expose raw pointers through the entire codebase and document how to use them correctly, when they can be mutated, who's responsible for freeing them, etc. In Rust, the natural pattern is to implement wrapper types and other abstractions that themselves make heavy use of unsafe blocks but whose APIs are safe.

This isn't unique to kernels. The standard heap-allocated vector type in Rust (on which the standard string type is based) is full of unsafe blocks: https://doc.rust-lang.org/src/alloc/vec.rs.html The problem of implementing a heap-allocated vector, with some elements uninitialized but available for use and with the ability to reallocate the entire vector and copy the members, is not a problem that lends itself well to the borrow checker's view of the world, so raw pointers are a fine way to solve it. Use of vectors themselves, their iterators, references to items in the vectors, etc. does match the borrow checker's view of the world, so the average Rust program running in userspace doesn't have to care that the standard library itself isn't implemented in safe Rust.

So copious usage of the "unsafe" keyword in a codebase can be a sign that the code as a whole is more safe: instead of having large portions of the code be unsafe (or instead of having the code pretend it's containing unsafety when it's not), only the parts that need raw memory access use it, and they maintain invariants used by the rest of the code.

It is certainly possible to do better - there are no machine-checked proofs that unsafe code does what it claims and upholds the invariants it claims to uphold. There could be, and I'm really interested to see what people do with languages that lend themselves better to proofs (usually dependently-typed, Turing-incomplete-but-very-powerful languages). But right now the state of the art is that such languages are a pain to use and writing proofs is even more of a pain. I think that's where the compromise is right now. The borrow checker is, arguably, a system for proving things about your code; it just only knows how to prove specific things. (Way more than C's type system proves, but way less than you'd prove in a perfect world.)

steveklabnik26• 9 years ago

You'd be surprised at how little unsafe you need. Consider https://os.phil-opp.com/page-tables/ for example, it uses the type system effectively to eliminate some issues, in the end, it only has two calls to unsafe.

On some level, yes, it's the best we can do. Unless you want to add hardware semantics into your language. Unless you do, writing two bytes to 0xB8000 is going to be an arbitrary memory access. At least, that's my take.

traverseda• 9 years ago

Having never used rust and having barely used C, here are some uninformed opinions.

A memory safe language for the kernel doesn't seem possible using "traditional" safety tools. I'd expect a "safer" language would look a lot like formal verification. If you can make a safer language that doesn't look a lot like formal verification, you can probably automate a lot of formal verification.

There are still some big advantages to rust. By explicitly marking unsafe code, you've made it a lot easier to figure out what needs to be focused on. I imagine you can even formally verify just the unsafe parts of your ecosystem.

Also, rusts tooling just seems more pleasant.

013a• 9 years ago

Even if you wrote your entire program in unsafe, you're still enjoying an experience at least as good as C++; you just don't get the compile time memory safety. And its unlikely your entire program would ever have to be in unsafe.

Thaxll• 9 years ago

Rust is not really good for web backends, it has no library for talking to popular service, the construct for networking are still uncertain / unstable / hard to use.

bluejekyll7• 9 years ago

For reference, what I have used that makes me claim this without being misleading (not necessarily all at the same time):

https://rocket.rs/, https://rusoto.github.io/rusoto/rusoto_core/index.html, https://docs.rs/postgres/0.15.1/postgres/, https://docs.rs/kafka/0.7.0/kafka/, https://docs.rs/reqwest/0.8.1/reqwest/, as well as many other supporting libraries. I've not run into many things where there isn't already support.

Also, in terms of networking, if your doing blocking IO, the stdlib is fine. Otherwise, from my experience, I really like https://tokio.rs/ for non-blocking (I know people are torn on this one) and have used it in my DNS impl, https://github.com/bluejekyll/trust-dns

For more info on web and Rust, this is a great page: http://www.arewewebyet.org/

justincormack• 9 years ago

I think another contender here would be OCaml/Reason, which has both JavaScript and WebAsm targeting, is used in systems space (eg Mirage). Its a bit different in technical choices (functional, GC, for example). Facebook is using Reason for Messenger for example, on mobile.

d215• 9 years ago

Honest question, Is reason really targeting wasm right now? Do you know a resource describing this? I cannot find anything about this.

idibidiart2• 9 years ago

Any links to instructions for Reason WASM targeting?

smt88• 9 years ago

I consider Rust to be functional, although people would dispute that, I'm sure. GC is a huge difference though.

shams93• 9 years ago

There is a project TrueBit is working on for ocaml and wasm integration.

verletx642• 9 years ago

Rust is great for the embedded space

Honestly, I would disagree here, unless we're talking in terms of potential, rather than actual ability right now. There's certainly some level of support, but I really did not get the feeling that Rust was 'great' for embedded, rather that it could be.

bluejekyll7• 9 years ago

but I really did not get the feeling that Rust was 'great' for embedded, rather that it could be.

My experience is very limited here, but I have used TockOS on one of their dev boards. That was very pleasant, no stdlib makes it a little harder than std Rust, but still felt more deterministic than writing in C.

ChickeNES• 9 years ago

Agreed, I've been waiting 3+ years for first-class support for bit fields

koffiezet• 9 years ago

Same, I have a number of devices here on my desk that will never get a Rust compiler targeting them. With C, it's simple, the "only" thing you have to worry about is what bugs or quirks the C compiler will have on that specific platform, not if there's a compiler at all.

Now I've left embedded development behind me for a while now, things are mostly Linux on relatively modern ARM in the field where I used to do stuff, so Rust would probably be an option there, but I still have a hard time considering them to be "embedded".

FrappucinoFill• 9 years ago

which can scale for every potential use case

For a certain type of lowish-level programming, yes. But there's a reason that many projects include something like Lua for the majority of the actual application-level work.

rapind• 9 years ago

Nim comes to mind. https://nim-lang.org/

dom96• 9 years ago

Thank you for mentioning Nim here. :)

To expand a bit, Nim features its own JS backend with a high proportion of the stdlib supporting it. You can access the DOM via the `dom` module[1] and build some pretty cool stuff[2][3]. It doesn't target WebAssembly (yet) however.

On the systems side, Nim compiles to C/C++/ObjC and manages memory via a soft real-time GC (or via a choice of other GCs including boehm).

1 - https://nim-lang.org/docs/dom.html

2 - https://nim-lang.org/araq/karax.html

3 - https://picheta.me/snake/ (source available here: https://github.com/dom96/snake)

santaclaus• 9 years ago

Is gaining traction in the native application space

Are there any Rust-first UI frameworks rivaling something like Qt, yet?

executesorder66• 9 years ago

Does gtk-rs count?

http://gtk-rs.org/

https://github.com/gtk-rs

stochastic_monk• 9 years ago

Modern C++ with RAII checks all of your boxes except for web backends. (Though I've heard good things about WT.)

Disclosure: I only write software for scientific computing and have no interest or experience in web development.

knowThySelfx• 9 years ago

Great. I like the way Rust is going on about.

phkahler2• 9 years ago

So when will servo become a webapp so we can run it inside firefox?

To clarify, this is sarcasm. What I really want to say is when can we run firefox inside firefox? But it's not entirely written in Rust. The point of that question is to illuminate some of the absurdity of web apps. i.e. what function does the outer browser serve in this scenario and why do we need it?

Edit: All the answers point to the outer browser as providing a sandbox. I contend that is supposed to be an OS function. I also think that tabs in the browser came about because OSes didn't provide a decent way to handle multiple instances of applications. See what's happening here?

jonreem• 9 years ago

It still serves as a sandboxed interpreter/GUI framework and as a frontend for finding and loading apps, regardless of how complex the apps are.

a_humean2• 9 years ago

Isolation. WASM is a target that allows users to effortlessly download and run arbitrary code from the web on all platforms with a modern web browser; an environment that is sandboxed such that arbitrary file system access or other system access is relatively safe and limited even without knowing the intentions of the author.

This is why javascript is so popular with its current monopoly on the web. No other language can do that, but with WASM other languages, such as Rust, can be part of the picture.

dullgiulio2• 9 years ago

I don't want to attack Rust or WASM, as I am pretty excited about this, but it's not that easy.

That anything that runs into the browser is isolated sandboxed and whatnot is irrelevant. It's still arbitrary code running on often unwilling clients.

Don't believe me? Think about ad-blockers.

geofft2• 9 years ago

It serves the same purpose as the outer kernel on a VM host: isolation between fully-featured inner environments using a solid, featureful platform. There are tinier platforms you can use for isolation, and yes, they're more secure, but they don't get you the wide real-world deployability that full-featured kernels or browsers get you.

I would genuinely like to be able to run Servo inside my un-jailbroken Chromebook. I already run an SSH client in the browser (using Google's Native Client instead of wasm, but same difference), and I don't see why Servo needs access to my SSH private keys. To me, the fact that the only way I can run Servo is to give it access to my entire user profile is the absurd thing.

bluejekyll7• 9 years ago

Your question is related to the technical aspects of the ecosystem, but not to the way that people use computers in general.

A good rhetorical question in response is, why is the web so much more popular when using computers than installed applications?

The implication you've made is that there is no benefit to the web, but that is obviously not true, as web usage far outstrips installed applications (besides the browser) at this point. Given this, it becomes necessary to have common tooling to target the different browsers out there.

Getting back to the core of your question, if there was a framework for which you could target all users on the internet (with a single codebase), it ran natively on all platforms, was easily run by everyone merely by clicking on an icon, which then cleans itself up after running... then maybe native apps would be a good avenue for delivering all applications.

qznc• 9 years ago

What is the advantage of wasm over Java and Flash? I don't really see technical advantages. There is only the social/legal aspect: wasm is not controlled by a single corporation.

Jyaif• 9 years ago

You can already run ff in ff:

https://bellard.org/jslinux/vm.html?url=https://bellard.org/...

chrismorgan• 9 years ago

Windows 2000? Then you can only run up to Firefox 12. (And Rust code came to Firefox a few years later.)

mrdoops• 9 years ago

I feel like the direction web and applications in general are heading is that of near-native performance in a sand-boxed execution environment managed by something like a browser. This potential is what excites me the most about webassembly, even more so than enabling the transition away from Javascript.

mr_toad• 9 years ago

I contend that is supposed to be an OS function

Different sandboxes for different needs. Processes, Jails, Docker, Hypervisors/VMs, Interpreters, they virtualise different resources with different levels of abstraction, and they all have pro's and cons.

sp3322• 9 years ago

It's in active development. https://github.com/browserhtml/browserhtml This way you don't have to depend on the browser including certain functionality because you can fall back to downloading a whole rendering engine.

zaarn7• 9 years ago

It would certainly become interesting if Firefox was largely just a WASM interpreter with OS extension in which a Browser Engine runs, written in Rust, compiled to WASM. Though I imagine to get proper sandboxing I'd probably implement a Ring System similar to x86 Rings for privilege seperation.

steveklabnik26• 9 years ago

I'm so so so excited to see this land! I have a little hello world demo: https://gist.github.com/steveklabnik/d86491646b9e3e420f8806a...

To give you an idea of file sizes, the above produces a 116 byte wasm file. If you leave off the no-std stuff it produces a 3059 byte wasm file.

EDIT: I'm actually being told that there was a bug in wasm-gc that's since been fixed, even with libstd it should be able to eliminate all of it and get the tiny file now.

josephg4• 9 years ago

That’s gorgeous. How much bigger does it grow if you heap allocate? Does it include 2mb of extra code like in normal desktop builds?

steveklabnik26• 9 years ago

I’m away from my computer right now, so may take a while to give you an exact number, but when I first did this, I forgot to turn on optimizations, and the libstd version was 173kb pre-gzip. So I’d imagine it’s similar to that, or at least, that gives you an order of magnitude.

killercup2• 9 years ago

This Rust code [1] heap allocates some things (e.g., a string with `String::from`), and results in this [2] WASM file, which is 65kB, or 25kB after `gzip -9`.

[1] https://github.com/killercup/wasm-experiments/blob/bf3b30eed...

[2] https://github.com/killercup/wasm-experiments/blob/bf3b30eed...

sinhpham2• 9 years ago

Could you please add an example of using cargo instead of calling rustc directly? How do I configure cargo.toml?

steveklabnik26• 9 years ago

Here's hello world: https://github.com/steveklabnik/semver.crates.io/commit/dc3b...

(I wanted to try to port semver parsing to the web today, but I hit an LLVM assertion, so it'll have to wait until we can fix that issue. This stuff is still very raw!)

xstartup2• 9 years ago

We switched from Go to Rust in production. After the initial ramp-up time, it's really a lot more productive than Go for us, at least.

qualitytime• 9 years ago

May I ask what "production" is?

zimzim• 9 years ago

the actual product environment, not development research or testing.

xstartup2• 9 years ago

Real-time fraud detection as service for adtech companies. We started with Java but got tired of GC pauses, perhaps due to an inexperienced team but we were not able to tame it. Then we moved to Go and now we are in Rust.

johnydepp• 9 years ago

Interesting! Can you tell what are you working on? It may help others.

__abc• 9 years ago

Asking for the next Pirates movie? (couldn't resist)

FourSigma• 9 years ago

Ditto! Would love some clarification.

stmw4• 9 years ago

We've been happy with Rust, as well.

drhodes2• 9 years ago

Has anyone given any thought to dynamic linking wasm objects to save bandwidth?

steveklabnik26• 9 years ago

It totally works, it's just not particularly easy to do. wasm modules have import and export sections; you'd instantiate the std module first, the instantiate the module that depends on it, pointing its imports at std's exports. I don't know of any tooling that makes this super easy at the moment though, you basically have to do it by hand as far as I know.

kodablah2• 9 years ago

I don't think you can at runtime from wasm though. Imports are resolved at load time. Sure there are call indirect that let you dynamically change the function index but not the import/export. Now if you were willing to reload, you could use the JS API to create modules at runtime with dynamically downloaded pieces, but once created, that's it I think.

drhodes2• 9 years ago

great, thanks, that clarifies things quite a bit.

ainar-g• 9 years ago

I would prefer compilers that produce compact statically linked "binaries".

bluejekyll7• 9 years ago

Assuming that the Rust stdlib eventually is compatible/compiled for WASM, wouldn’t it be better if that was cached for all sites which use it, rather than downloading it each time?

I can imagine a world where all dependencies are compiled for WASM, similar to crates.io, and then all apps using those libs would benefit from sharing and load blazingly fast because they only load the site specific code.

Wouldn’t that be a good thing?

doiwin• 9 years ago

I would like to code WebAssembly by hand. Are there any good tools / tutorials on this?

kodablah2• 9 years ago

Probably not any good tools or tutorials, but the wast format is not that hard [0][1] and most compilers accept that format. Just gotta keep the values on the stack in mind while you write.

0 - http://webassembly.org/docs/text-format/

1 - https://webassembly.github.io/spec/text/index.html

lj3• 9 years ago

Do you mean something like this? https://mbebenita.github.io/WasmExplorer/

idibidiart2• 9 years ago

How does the performance of WASM compare to JIT'd JS code?

amelius• 9 years ago

This makes me wonder: does WASM allow a JIT (specializer)? I.e., does WASM allow self-modifying code or code to write code?

If not, then JIT'ed JS could still be faster than WASM.

steveklabnik26• 9 years ago

Wasm code is normally JIT-ed by the JS VM. That is, browser implementations don’t have an entire separate VM for wasm.

amaranth• 9 years ago

It's going to be heavily dependent on what you're doing. WASM can be as fast as native code but is more likely to be 1/2 the speed of native on realistic benchmarks. However, the overhead of moving between WASM and JS is quite large so in practical use your WASM version might be slower than a pure JS version. WASM vs JS benchmarks I've seen show WASM anywhere from 40x faster than pure JS to 10x slower than pure JS, due to this overhead.

If you're doing a large amount of CPU intensive work that can be isolated from the rest of the code that is an ideal scenario for WASM. You have the translation overhead of JS->WASM once, then do the computation, then WASM->JS once to get the result. A small function or code that needs to call back into JS (for DOM access, to query some state in your app, etc) is going to have a lot of those translations so won't see much or any benefit.

steveklabnik26• 9 years ago

You might enjoy this thread https://www.reddit.com/r/programming/comments/7fm66l/astro_p...

JepZ4• 9 years ago

Actually, I don't know why so many people are so excited about webassembly. Yes, it is faster than JS, but JS itself is not that slow and on the other hand we loose the open nature of the human readable web.

Kwastie• 9 years ago

Minifiers don't output any readable JavaScript anyway. Yes, it can be "decompiled" back to something readable, but so can WASM.

JepZ4• 9 years ago

Actually, with minified JS you do not have to "decompile" anything. It is just pressing the pretty print button in your dev-tools and you can use your normal debugger (just without propper names) to step through the code.

poizan42• 9 years ago

Is it even faster? There is no particular reason why asm.js should be slower besides the initial parsing time.

flohofwoe2• 9 years ago

You are absolutely right (don't know why you are downvoted). On most browsers, WebAssembly is only a tiny bit faster than asm.js (except Safari on iOS11 where wasm is about 3x faster, but this mainly because their Javascript performance is so bad compared to the other browsers). But that just shows how good asm.js already is. The only major thing that WebAssembly currently has over asm.js is proper support for 64-bit integers.

The "initial parsing time" advantage of WebAssembly shouldn't be underestimated though, this is important for bigger code bases, and WebAssembly binaries are also a bit smaller (after compression) than the same code as asm.js.

steveklabnik26• 9 years ago

WASM has a non-lossy text based format; everyone involved doesn’t want to lose view source.

JepZ4• 9 years ago

So and that source is as easy to read and step through as minified JS (didn't know something like that exists)?

And does the one who deployed the code has to use the non-lossy text format or can you simply convert any WASM binary to something human readable (with browser dev-tools)?

borplk• 9 years ago

This is a really tired meme. It's not any less accessible than minified js.

souenzzo• 9 years ago

Once rust is implemented in rust, I can run rustc in browser and get binaries for windows, for example?

steveklabnik26• 9 years ago

Rust is written in Rust, with the exception of LLVM. C++ can be compiled to wasm too though, so that's not the blocker.

The bigger issue is that rustc uses features that aren't in wasm yet, like threads and file loading and such. It'll happen eventually, but not just yet...

stcredzero2• 9 years ago

What is the Rusty way of handling memory management for a game engine client deployed on WebAssembly? When a program is running a game loop, there are certain entities which will stick around across frames, and there are certain entities which definitely will not. Is there a way to enforce that most of the implementation of the game reside in pure functions, while making memory management nearly bulletproof?

steveklabnik26• 9 years ago

Nothing about this is specific to WebAssembly; you’d write the same Rust code as you would for any other target.

stcredzero2• 9 years ago

In that case, what's the Rusty way to handle a game loop, in which one wants to encourage the use of pure functions? I must confess: I'm thinking of using Rust and a few other languages as a compiler target for a unified Web Client + Server game engine. Is there a particular way in which Rust could support this better than other languages compiling to WebAssembly?

api• 9 years ago

Rust glued to React Native or the latter actually ported to Rust sounds like Nirvana.

fleetfox• 9 years ago

And webrender as rendering layer for desktop apps to replace electron

substack• 9 years ago

And there is already a browserify transform to compile inline rust to webassembly for browser code: https://github.com/browserify/rustify

alex_duf• 9 years ago

Such a great news!

I'm looking forward DOM integration with wasm, and potentially a way to compile from scala native to wasm.

Rust: Enable WebAssembly backend by default

Related Stories