Superfast asm.js benchmarks of V8 TurboFan

http://arewefastyet.com/#machine=28&view=breakdown&suite=asmjs-ubench

by Pharohbot • 12 years ago

37 30 12 years ago

arewefastyet.com

desdiv• 12 years ago

How the hell did C++ lose to Javascript on the first benchmark!?

And it's not just a little slower either, it's 36% slower than the fastest Javascript implementation.

stingraycharles• 12 years ago

For those interested, here is the C++ code of the first benchmark:

https://github.com/h4writer/arewefastyet/blob/master/benchma...

ridiculous_fish5• 12 years ago

Oh yeah, I love this stuff.

On my Linux box, clang 3.4 is 50% slower than gcc 4.9 on -O3 on that code, for all inputs.

The key difference is, unsurprisingly, in the inner block: for (int j = 0; j < 50000; j++)...

clang's inner block asm: https://gist.github.com/ridiculousfish/24b16ba5b4e8abbe051b

gcc's inner block asm: https://gist.github.com/ridiculousfish/bee5217d6ae1f4f34a5d

Notice that clang's is much longer. Also notice all of the "# imm = 0xFFFFFFFFFFFFFC00" lines in clang. Those immediate values correspond to inlining the norm() functions: it's how clang is computing x%1024.

How do you compute x%1024? It sure would be nice to strength-reduce it to x&1023, but that's always going to produce a non-negative number, which is not what you want if x is negative. That is, -1025 % 1024 ought to be -1, and not 1023. So that strength reduction (x%1024 -> x&1023) is only valid if you can prove that x is non-negative.

And that's the first big difference. gcc-4.8 convinces itself that all of the ints involved are non-negative, and so it computes x%1024 as x&1023. clang fails to notice that all ints are non-negative, and so emits the slow path for the mod.

Let's replace int with unsigned int, and see what happens:

clang's uint asm: https://gist.github.com/ridiculousfish/c35ffcff5963160c9622

This allows the x&1023 strength reduction, but now we hit the second big difference: clang is futzing with xmm registers, perhaps packing a single 'vec' into a register. The codegen just looks silly to me - all those pshufd and punpckldq. Let's disable that noise with -mno-sse:

clang's uint asm (no SSE): https://gist.github.com/ridiculousfish/0afc79d458ae30dc6327

Now clang is totally kicking ass, with fewer instructions than gcc, and about 17% faster. (Of course gcc may in turn be massaged to be even faster yet.)

Timings on my i3 NUC with -O3, input 4. Lower is better:

gcc w/ signed: 4.78 sec

clang w/ signed: 7.11 sec

gcc w/ unsigned: 3.83 sec

clang w/ unsigned: 7.11 sec

clang w/ unsigned no-sse: 3.14 sec

In summary, clang is missing a big optimization, and then emitting some dubious SSE code. Fixing those more than doubles its speed. This illustrates the importance of monitoring your codegen for tight loops, and the hazards of micro benchmarks!

vardump11• 12 years ago

Plain C++ can easily lose to JITted scripting languages. Nothing new or weird in that. Remember, we're comparing native code to native code.

accurrent• 12 years ago

Not true actually theres always a JIT overhead for the compiler.

joakleaf• 12 years ago

Easily?

Do you have any benchmarks to back that up?

My sense is that JITet platforms lose most of the time still, but that the JITted platforms are getting closer, and occasionally win.

ridiculous_fish5• 12 years ago

Yes! But it's important to keep in mind the sort of optimizations that JIT enables: monomorphic dispatch, type specialization, etc. And also what it's bad at: computationally expensive or memory-intensive operations, like graph-coloring register allocation.

So generally speaking, we'd expect JIT to shine with certain branchy or unpredictable code, but static compilers to shine for basic blocks. Since the hot spot of the code in question is one big basic block (no if statements, etc.), where even the loop trip count is known statically, it would be surprising in this case if a JIT outperforms a static compiler.

detrino5• 12 years ago

Two things to keep in mind:

1) This program has no negative numbers yet the c++ source uses signed ints while having a lot of modulo calls which are slower than if unsigned int was used

2) This program was benchmarked with clang++

In the following numbers "new program" is replacing signed int with unsigned int and all times are measured with perf stat -r5 <binary> and run on an old intel wolfdale cpu.

Original program compiled with clang++ -O3: 1.441151117s

Original program compiled with g++ -O3: 0.826723649s

New program compiled with clang++ -O3: 1.181305003s

New program compiled with g++ -O3: 0.721448016s

Edit: Newer program with proper pass by reference vec::add

Newer program compiled with clang++ -O3: 0.770128848s

vardump11• 12 years ago

Yeah, same silly things programmers do in real world statically compiled code. Bad choices for something like data type and bad compiler options. A JIT can figure out negative numbers can't happen and optimize accordingly. It might know about the CPU model that didn't even exist when the static code was compiled.

azakai• 12 years ago

1. That is on 32-bit, for fairness it's worth mentioning that Safari is slow there because its FTL JIT is 64-bit only.

2. Firefox would also be faster on 64-bit as well (less significantly). However, TurboFan results aren't showing up on the 64-bit bot - not sure if that's an issue with the bot, or if TurboFan doesn't build on 64-bit yet.

3. Those are microbenchmarks, for more realistic workloads click on "apps", which leads to http://arewefastyet.com/#machine=28&view=breakdown&suite=asm...

zobzu• 12 years ago

Looks more like firefox + ion monkey is superfast on asm.js, not chrome

which.. actually reflects reality.

cromwellian• 12 years ago

Is Turbofan even beta yet? Seems a bit early, it performs worse than crankshaft on some code at the moment.

Superfast asm.js benchmarks of V8 TurboFan

Related Stories