Replacing JavaScript Hot Path with WebAssembly

https://developers.google.com/web/updates/2019/02/hotpath-with-wasm

by markdog12 • 7 years ago

226 103 7 years ago

developers.google.com

Roboprog2• 7 years ago

Too bad no wasm on IE 11

Where it would be needed most

tracker1• 7 years ago

Depends on your use case... a lot of places are just plain deprecating IE altogether.

ghayes• 7 years ago

Isn't Microsoft one of the founding companies in the initiative? [0]

[0] https://webassembly.org/

Roboprog2• 7 years ago

As well they should:-)

But IE 11 has been given a stay of execution until 2025 I believe.

And sadly most of my users are still on it (government and healthcare)

iamleppert• 7 years ago

Did you see the benchmarks? There's almost no difference between javascript and wasm except for a single certain browser. So you're really going to take on the maintenance burden to get that better performance?

This is a cool technique but I can just imagine the looks on my team mates faces when I tell them it isn't react... :/

nicoburns• 7 years ago

Weird that they didn't say which browser is which :/

RivieraKid2• 7 years ago

Probably were worried it would look like they're trying to shame their business partners (Apple and Microsoft I guess).

fireattack• 7 years ago

They said it in the article:

Note: Due to legal concerns, I won’t name any browsers in this article.

ekc• 7 years ago

That's not true in almost any circumstance. What "benchmarks" are you referencing?

gbba• 7 years ago

Yes if it provides a better product for users.

bitwize• 7 years ago

There's got to be at least one RESF member on your team. You can sell the technique by telling them they get to use Rust in your web front end.

mnutt• 7 years ago

It isn't a rebuttal to this article, but I found this blog post about optimizing javascript to be really, really interesting:

https://mrale.ph/blog/2018/02/03/maybe-you-dont-need-rust-to...

markdog123• 7 years ago

Don't forget to follow that up with the second follow-up: http://fitzgeraldnick.com/2018/02/26/speed-without-wizardry....

writepub2• 7 years ago

WASM's biggest claim to fame is providing web development access to non js devs. Having done C for a majority of my life, the ability to build and execute C code for large scale web deployment is appealing!

tom_mellior2• 7 years ago

There's almost no difference between javascript and wasm except for a single certain browser.

For very large values of "single", approaching "two". In the "Speed comparison per language" chart, Browser 3 is more than 5x slower than Browser 2 on JavaScript/WASM, and Browser 4 is slower still. So there are very significant improvements on two out of the four browsers tested.

maeln• 7 years ago

We have to remember that the current WASM spec is still "just" a MVP. It doesn't yet include performance related spec (like SMID). WASM is also fairly recent. JS interpreter/JIT in browser has seen years of optimization with a trove of real world usage. It will take some time for WASM to be able to compete seriously.

Another factor is also that the WASM compilers for various languages (Rust, C/C++, etc) are obviously recent too and not super optimized.

My own tiny experiment is that WASM can already yield quite decent performance gain but with very compute intensive load, which is not a typical problem in frontend development. The size gain is also real, but you need to handcraft your WASM or forget about using the std and other stuff in the language you are compiling from (Rust generate very fat binary with a naïve implementation for example).

Still, I am quite optimistic about WASM. I was actually impressed that, even though it is quite recent, I can already compete with JS when it come to performance. When the various performance-related spec will be finalized and implemented and that browsers and compilers start heavily optimizing the WASM, we should really see some real-world gain.

PhilippGille• 7 years ago

I would have loved to see Go included in the comparison as well. It can compile to wasm since 1.11.

dassurma3• 7 years ago

Go was pretty much a non-starter. They (currently) need a runtime which will make the file size non-competitive to the other ones. Also, since only Chrome has support for threads in WebAssembly (in Canary), we’d not be able to make any use of the concurrency.

asien• 7 years ago

The performance gain are so small , its not worth this setup overhead . The average user won’t see the difference. Hence , the simplicity of’this module . Just do it in JS , chrome as 70% market share why would you ever bother ?

V8 has received decades of optimizations and it can easily compete with compiled languages in terms of speed.

I was hyped to death for WASM , but this is the tenth article I’m reading on this subject and I still ending on the same conclusion : there is no advantage for front end developers to use WASM.

Only Rendering Engine ( Unity , Adobe Products, Autodesk ) can really benefits from this.

writepub2• 7 years ago

The article sets out to prove the predictability of WASM's performance, and not necessarily a performance gain wrt js.

This confirms what we laid out at the start: WebAssembly gives you predictable performance. No matter which language we choose, the variance between browsers and languages is minimal

If you're not hyped about WASM, it's probably because your app and customer base's browser preferences are on the js engine's JIT happy-path, which could hold true for most apps. There could very easily be a js path that is significantly worse in performance on chrome, just saying, 70% market share is both a blessing and a curse.

Another major reason for WASM hype is for C#, Rust, C, C++, Go devs to reach parity with js in terms of web accessibility. Frameworks like Blazor (from MSFT) have taken all the best practices & advantages of React and made them available to C# devs.

pjmlp3• 7 years ago

have taken all the best practices & advantages of React and made them available to C# devs.

The irony is that C# devs were the first to use reactive programming before React even existed.

news_to_me• 7 years ago

chrome [h]as 70% market share why would you ever bother ?

This view seriously needs to die. It's honestly not that hard to test in two or three browsers, and the differences are minor enough that it isn't a pain. But the only way that's possible is through Web standardization, which only happens when there are diverse options.

As web developers, it's our duty to keep the web healthy, and that means not only optimizing for a single browser.

outworlder• 7 years ago

Agreed. This is IE 6.0 all over again.

devwastaken• 7 years ago

Chrome hasn't optimized wasm very well yet. Wasm isn't and never was meant for frontend. It's meant for crunching data and making it possible to use the thousands of C libs computers run on to have a safe and efficient execution environment that is not restrained by the host software having implimented that C lib directly.

For example there was this app in C# that would convert images into 512 color palette and use dithering to retain some quality. I made a version in the browser, but because of js being too slow it didn't work for large images. Thing is, mine was far safer and accessible than the C# program.

gok• 7 years ago

Wow all these numbers seem insanely bad. 500 milliseconds to transpose 16 million pixels (so 64mil bytes)? A modern CPU should able to do that at least 10x faster, if not 100x.

robko6• 7 years ago

You are correct. The code is using an inefficient cache access pattern, so most of the time is spent waiting.

You probably won't get 100x faster without SIMD, but 10x is certainly doable. Unfortunately, SIMD.js support has been removed from Chrome and Firefox a while ago, even though it is not available in wasm to this day.

kllrnohj2• 7 years ago

How would SIMD do anything to address the problem's fundamental anti-cache-friendly access patterns? You'd need to restructure the problem to be cache-friendly, but SIMD won't really be relevant to that.

kllrnohj2• 7 years ago

They are bad but not way off for that basic for loop, depending on which rotation is being applied.

Using their code on my Intel-based workstation at around 3ghz using GCC 7.3 it takes around 80-100ms to rotate a 4096x4096 buffer 90 or 270, and 14ms to rotate 180.

Max memory bandwidth of something like an i9-9900k is 41.2GB/s. This test reads & writes 128mib of data. So max theoretical achievable performance here is around 3-4ms. Max theoretical. So 100x is not really feasible. 10x, though, very much is, as the quick convert shows a peak time of 14ms with a 180* rotation.

Of course the major source of slowness here is that the reads/writes are not sequential, and the 90 & 270 rotations are achieving a fraction of the possible bandwidth they could as the input reads are jumping around, so every single one is a cache miss and the other 60 bytes in each cache line on the miss will be purged before it's used again.

Flipping it would mean the writes are never utilizing a full cache line, either, though. So you can't really "fix" that, not easily at least. So either your read or write bandwidth ends up tanking and you can only achieve roughly 6% of the max (only ever using 4 bytes of the 64-byte cache line) for that half of the problem. Without some clever magic to handle this your max theoretical on a 41.2GB/s CPU drops to around 50ms.

All that said it's clear that WASM is very far off from native levels of performance. ~5x slower isn't something to brag about. But hey maybe the test system was a potato, and the 500ms isn't as bad as it sounds.

bufferoverflow2• 7 years ago

Or better yet, WebGL should be able to do this in a few ms on a GPU.

olliej9• 7 years ago

Or simply use the canvas api, which has super optimized graphics libraries behind it - rather than reimplementing the wheel :)

But I get that really this was a how much can wasm help performance as % vs js - you could always write an “optimized” routine and compare those and theoretically achieve something similar.

robko6• 7 years ago

My guess is that the JS implementation of the worst-performing browser is having trouble with the non-1 for-loop steps. Doing 90-degree image rotation with fixed steps and some index calculations should work better (0.18 sec vs 1.5 sec for their implementation in node.js):

    for (var y = 0; y < height; y++)
        for (var x = 0; x < width; x++)
            b[x + y*width] = a[y + (width - 1 - x)*height];

Although that's still far from the theoretical maximum throughput because the cache utilization is really bad. If you apply loop tiling, it should be even faster. This problem is closely related to matrix transpose, so there is a great deal of research you can build upon.

EDIT: 0.07 seconds with loop tiling:

    for (var y0 = 0; y0 < height; y0 += 64){
        for (var x0 = 0; x0 < width; x0 += 64){
            for (var y = y0; y < y0 + 64; y++){
                for (var x = x0; x < x0 + 64; x++){
                    b[x + y*width] = a[y + (width - 1 - x)*height];

seanwilson2• 7 years ago

Maybe this is what you meant but the snippet can be optimised a ton as well unless I'm missing something:

- Move the "y * width" calculation outside of the "for x" loop.

- The multiply operators can be replaced with addition e.g. replace "y * width" with "counter += width" each y iteration and similarly for the x loop.

Optimising inner loops is really fun.

How much of the speed up in the article is because the JS engine can't figure out how to optimise it compared to the WebAssembly compiler?

robko6• 7 years ago

I tried a few micro-optimizations, but they did not make a measurable difference, so I kept the code short instead. But maybe some JIT is particularly bad at loop hoisting, so it might make a difference there.

tom_mellior2• 7 years ago

These code motion/strength reduction optimizations are standard even in mildly optimizing compilers. I would be very surprised if an optimizing JavaScript compiler did not perform them automatically.

acqq6• 7 years ago

Your 0.18 sec result is (to use the units they used in the article) 180ms, and if I understand correctly their best webassembly compiled and executed result (?) is 300ms. Beautiful.

EDIT: But it could also be that your computer is somewhat faster than theirs? Do you happen to have some very fast CPU? Can you say which? When I run C-like C++ versions of your code I get the speeds you get with node.js. However, you made overall much better results than they were able, it's still great work!

    #include <stdio.h>
    int main(int argc, char* argv[]) {
        enum { height = 4096, width = 4096 };
        unsigned* a = new unsigned[ height*width ];
        unsigned* b = new unsigned[ height*width ];
        if ( argc < 2 ) { // call with no params
            // to measure overhead when just allocations
            // and no calculations are done
            printf( "%d %d\n", (int)a, (int)b );
            return 1;
        }
        if ( argv[1][0] == '1' ) // call with 1 the fastest
        for (unsigned y0 = 0; y0 < height; y0 += 64)
            for (unsigned x0 = 0; x0 < width; x0 += 64)
                for (unsigned y = y0; y < y0 + 64; y++)
                    for (unsigned x = x0; x < x0 + 64; x++)
                        b[x + y*width] = a[y + (width - 1 - x)*height];
        else
        for (unsigned y = 0; y < height; y++)
            for (unsigned x = 0; x < width; x++)
                b[x + y*width] = a[y + (width - 1 - x)*height];

        return 0;
    }

vijaybritto• 7 years ago

I think its fast because of the L1 cache or something like that. I dont understand fully but this is what i got

robko6• 7 years ago

I get 60ms in C. But in your code, the compiler might decide to remove most of the code since b is not used after being calculated. I checked the assembly code and it does not seem to be the case here, but it's still something to be aware of.

maxgraey• 7 years ago

https://www.reddit.com/r/programming/comments/aqpjkx/replaci...

seanwilson2• 7 years ago

As I understand they the main goal was to achieve easily readable and maintainable code, even to the detriment of performance.

Seems like a tricky goal for image algorithms in general where you're performing the same action over and over on millions of pixels. Obscure inner loop optimisations are pretty much required.

In these situations, I would sometimes keep the code for the naive but slow version around next to the highly optimised but difficult to understand version. You can compare the output of them to find bugs as well.

RivieraKid2• 7 years ago

My guess is that the JS implementation of the worst-performing browser is having trouble with the non-1 for-loop steps.

Why would non-1 for loop be slower in some browsers? Does the compiler add some sort of prefetch instruction in the faster browsers based on the loop increment?

yalok• 7 years ago

Can someone please explain why loop tiling increases performance in JS so dramatically? Is it mainly due to the fact that inner loops have constant size (64) and get called more frequently, and thus get promoted faster into deeper stages of JS runtime optimization?

My guess is that if you try to invoke initial whole code (before tiling) in a external loop (rotating images of exactly the same size), you will get similar perf boost (not that it has practical implication, but just to understand how optimization works).

vardump• 7 years ago

No, it's faster because the working set of 64 * 64 * 4 * 2 bytes can (almost) fit in CPU core L1 cache. Further cache levels are slower and finally the memory is glacially slow.

WASM example would speed up as well using the same approach. Or C, Rust or whatever.

dassurma3• 7 years ago

Huh interesting! I always disliked butchering code to do processor cache optimizations and I kinda worked under the impression that a browser’s JS and wasm compilers would do these optimizations for me.

I’ll definitely give tiling a spin (although at this point we are definitely fast enough™️)

bufferoverflow2• 7 years ago

I tested these two pieces of code in different browsers on i7-8750H with 16GB of RAM.

Chrome: 248 ms vs 93 ms

Firefox: 552 ms vs 93 ms

MS Edge: 7486 ms vs 6186 ms

IE: 9590 ms vs 9156 ms

These are some WTF results, to be honest.

thefounder• 7 years ago

Can we expect a day when WASM will be first class citizen in browsers (i.e like JavaScript) and not just a sidekick?

tachyonbeam• 7 years ago

Not anytime soon IMO because WASM still has to access browser APIs through the DOM, which is really built with JS in mind.

goatlover• 7 years ago

Why not anytime soon? VBScript and Dart where given the ability to access the DOM in IE and Chrome in the past.

markdog123• 7 years ago

It's coming: https://github.com/WebAssembly/host-bindings/blob/master/pro...

int_19h3• 7 years ago

HTML DOM is described in terms of IDL interfaces, complete with types. I wouldn't say that it's optimized for JS - indeed, that's why jQuery and similar were introduced. When WHATWG took over, they improved it specifically for better JS interop, but it's still straightforward to map to most statically typed languages.

https://dom.spec.whatwg.org/#infrastructure

https://heycam.github.io/webidl/

markdog123• 7 years ago

With threads, SIMD, GC, direct DOM access and more, tools like Emscripten and wasm_pack.....yes

https://hacks.mozilla.org/2018/10/webassemblys-post-mvp-futu...

https://www.youtube.com/watch?v=VsYL4Z9sRec

amelius2• 7 years ago

And don't forget support for lock-free programming (memory fence instructions), useful when you want to implement your own specific concurrent GC, for example.

kodablah• 7 years ago

I'd be tempted to hand-write the WAT for that. It's not that bad, much easier than dropping into a x86 asm block in C or something.

dassurma3• 7 years ago

I did :D Turns out compilers are pretty good at generating WAT.

z3t42• 7 years ago

I would try optimizing the JS before dropping down to webassembly. For example try replacing let and const with var as let and const in loops have to create a new variable for each iteration.

Etheryte• 7 years ago

That's not how let and const work at all, where did you get that impression?

z3t42• 7 years ago

Have you ever made a for loop using var only to have the variable point to the last value in the iteration ? And had to make a closure using forEach, function or self calling function ? With let you do not have to do that as a new variable is created for each iteration. Instead or reassigned when you use var.

alangpierce• 7 years ago

The "predictable performance" point applies not just to performance across browsers but also that you don't need to pay JIT warm-up costs. A while back, I ran some benchmarks on the same codebase in TypeScript and AssemblyScript and found that wasm was much faster than JS for short computations and often slower than JS when V8 is given multiple seconds to fully warm up the JIT:

https://github.com/alangpierce/sucrase/issues/216

So really, it depends a lot on the use case. In my case, it's often a short-lived node process that a user is directly waiting on, so compiling to wasm is probably useful. It also depends on what you're doing; some types of work (e.g. where you'd want careful memory management) are a lot harder for V8 to optimize from JS and can be expressed more nicely in AssemblyScript or another language that gives more memory flexibility.

why_only_15• 7 years ago

For that, it looks like unless you're running the same js on a really huge dataset webassembly will win (going from the second speed test). Even when you're compiling 50MB of JS with that thing, Wasm is 5% slower than JS, and when you're compiling 500KB (more typical) it's 300% faster.

gcb0• 7 years ago

(overly ironic) tl;dr "let's rewrite something in this new browser feature because the other browser feature we added last week is not supported anywhere and buggy in chrome"

jijji• 7 years ago

anyone like to take a guess what "Browser 4" is?

barrystaes• 7 years ago

Hmm.. the Google article stipulates:

WebAssembly on the other hand is built entirely around raw execution speed. So if we want fast, predictable performance across browsers for code like this, WebAssembly can help.

So i wanted to see how i could use WebAssembly in a React webapps. I found this SO question sees the opposite:

When running this [ WebAssembly] code in Chrome, I observe "pauses" that cause the app to be a bit jittery. Running the app in Firefox is a lot faster and smoother.

https://stackoverflow.com/questions/53584607/react-app-with-...

Same code, different browser, different performance. I'd love seeing a Google Developer answering that question in depth..

IshKebab2• 7 years ago

That looks like a bug, rather than an inherent characteristic of wasm.

sisu2019• 7 years ago

WebAssembly is a bit underwhelming to be honest. It feels like every week there is a new language that can come close to C performance meanwhile they've been working on WebAssembly for years and years and it can barely beat JS.

Shouldn't WA as a greenfield project with it's extremely basic memory model and lack of runtime or standard library be super easy to optimize?

After all, there is no point in having the bad ergonomics of assembly together with the awful performance of JS, right?

amelius2• 7 years ago

Those are runtimes in the second range. Are they doing that in a separate thread or do they block the UI? And how long does it take to transfer the data to that thread?

Replacing JavaScript Hot Path with WebAssembly

Related Stories