Benchmarking JS

http://mrale.ph/talks/goto2015/#/

by luu • 11 years ago

131 31 11 years ago

reimertz• 11 years ago

I found this very interesting since I'm working on something similar where I need to be able to measure how long certain operations take.

And I had no clue that js-engines has come this far, optimizing code during runtime this efficiently(if you ignore the bugs.. :)).

DannyBee3• 11 years ago

It's actually really well written

Some of the optimizations compilers/jits will perform would blow your mind.

LICM is a very simple thing to do.

There are more advanced things, like http://www.cs.rice.edu/~keith/512/2011/Lectures/L06CLZ-1up.p...

etc

That's just on the code movement side too.

Most good static compilers can symbolically evaluate even linear systems of equations/recurrences/etc that are solvable, and transform loops that calculate them into constants, etc.

mraleph8• 11 years ago

LICM is a very simple thing to do.

Yeah, it sometimes surprises me how much value the most simple optimizations have and how much they break people's attempts to measure performance.

There is another side to this medal which is best expressed in a quote I picked up from a relatively old paper:

"A survey of the literature on optimization uncovers many techniques that are hard to understand, harder to implement, more general than necessary and of marginal value. Our approach was to do the most profitable machine independent optimizations in the most profitable way" (A Portable Optimizing Compiler for Modula-2 by Michael L. Powel)

In some sense the more microbenchmarks an optimization breaks the bigger its impact on real world code is :)

CountHackulus• 11 years ago

LICM is indeed very simple to do, I wrote it for a JIT emulator I was working on a while ago. It turns out that it's not always fast. There's a bunch of weird cases that happen that can destroy your performance.

LICM is highly dependent on the quality of your trace recording or method inlining.

Not disagreeing with what you wrote at all, just wanted to provide an anecdote.

barsonme2• 11 years ago

Does anybody happen to have the video for the talk? I'd be much obliged. Just the slides leave a bunch of holes I want to fill in.

I can only find the videos from 2014.

mraleph8• 11 years ago

It was recorded but I don't know when it will be publicly released.

This talk is a combination of two separate talks I have given before plus two new examples, based on my recent endeavors.

Talks merged here are LXJS 2013 one[1] where I talk about microbenchmarking pitfalls and WebRebels 2014 one[2] where describe how bugs in VMs can affect benchmarking results.

If you have any questions about the slides just ask them here or send me an email --- I will try to answer as soon as I can.

[1] https://www.youtube.com/watch?v=65-RbBwZQdU

[2] https://webrebels.23video.com/crooked-mirrors-of-performance...

tosh• 11 years ago

Thanks!

barsonme2• 11 years ago

Okay, thank you! I enjoyed the slides and those talks will give me a lot of study material for tonight.

Thanks again!

randomfool• 11 years ago

Your talks always frighten me when it comes to VM's. I just want predictable performance- is that too much to ask? (Semi-serious question).

edit: to clarify. In C++ I write dumb but readable code knowing that the compiler will always perform specific optimizations. In JS it seems that there are many factors resulting in different optimization levels. How do I ensure that the VM does not execute my 'dumb but readable' code literally? Without that guarantee it seems that I have to frequently resort to optimized yet unreadable code.

bjorn2404• 11 years ago

Really enjoyed that LXJS 2013 video the first time I saw it highlighted on Dave Methvin's blog ("Please Stop the jsPerf.com Abuse!"). Hilarious and very informative - thanks!

voltagex_2• 11 years ago

From https://www.youtube.com/playlist?list=PLEx5khR4g7PJPfVDB1nUp... it looks like you might be waiting a while - 2014 conference was in May, videos went up early/mid August.

voltagex_2• 11 years ago

Adding more info: https://twitter.com/GOTOchgo/status/598489575481286656

inglor2• 11 years ago

Can always trust Vyacheslav Egorov to pump out super high quality stuff about v8.

His blog http://mrale.ph/ is full of hidden goodies like Object.create vs function constructor [1] and hidden classes vs jsperf [2]. Lots of good stuff on his SO too.

It'd be great if he (hi if you're reading this btw!) could write a bit about TF and the changes being made to v8 internally from an overview point of view.

[1] http://mrale.ph/blog/2014/07/30/constructor-vs-objectcreate.... [2] http://mrale.ph/blog/2013/08/14/hidden-classes-vs-jsperf.htm...

inglor2• 11 years ago

And, since you're reading this (in the other comments) I just want to point out that the presentations you're giving got a lot better (the deck). This one is a lot more witty than the old ones. Props on getting better at presenting - a big step from your old presentations and blog posts, the English is good too here :)

gnrlbzik2• 11 years ago

Agree : ) Deff. awesome experience watching him present. Such talks stick with you through out.

mraleph8• 11 years ago

Thanks, I am glad you like my blog :)

I might write something about TF but right now it's still not used to compile normal code and the team is just starting on stuff like adaptive optimizations, so it's a bit too early.

Also given that I don't work on V8 anymore I always think it's a bit unfair for me to do it --- somebody from V8 team should do it instead, cause my perspective is bound to be slightly different from theirs :)

z3t4• 11 years ago

You should never "optimize" JavaScript code, like objects vs arrays or methods vs functions. But instead try to optimize your algorithms, like leaving out square root when comparing witch line is longer.

I did optimize JavaScript in the past, but it always worked different in different JS implementations and could change in different releases of the same JS engine. So I gave up on it, and it actually makes me more productive not having to think about with method is faster.

fridek2• 11 years ago

That is simply not true. To write fast JS you need to understand the basics of the engine, like what a hidden class is, what exactly garbage collector does, how to keep your methods monomorphic or how types work in arrays. I was stunned when recruiting people to see how many just didn't understand why the prototype chains they build are actually creating a separated method instance per each object they allocate. You can see that easily when serious people suggest that an object constructor should return another object with methods on it. Sure, it's valid JS, it has it's advantages and compiler MAY be able to optimize it (last time I checked it didn't), but there will be a limit in what it can do and performance will be impacted.

You should perhaps not optimize trivial things like methods vs function, because there is no good underlying reason for them to be much different. You should know when to optimize and what. I love the longer version of Knuth's quote - "We should forget about small efficiencies, say about 97% of the time: premature optimization is the root of all evil. Yet we should not pass up our opportunities in that critical 3%.". The overall architecture, the way you write and organize your code, in my opinion, and in case of JavaScript is likely to be in the 3%.

But I also agree not caring too much about it improves productiveness. Most applications really don't need to worry about performance in such depth. Even when they do, it often boils down to keeping one or two core components optimized.

The only worry I always have about it, is that when you notice your code style and architecture doesn't play well with JS engines, it's often too late to change it. A friend called me a day before a prod release asking what to do about his program taking a huge chunk of memory for what the profiler described as a "compiled code". It was a large corporate application, with dynamic code loading. We digged into the framework internals just to notice there was no built-in way of deregistering a class once it's created. Since it didn't play nicely with the JS engine, there were many more hidden classes than business models in the system, and most calls were polymorphic. All this compiled code had to be kept somewhere. At this point there was no way he would replace the framework and even quickly fixing it was a challenge.

amelius• 11 years ago

To write fast JS you need to understand the basics of the engine

The problem is that there are many different engines out there.

Unless of course you are talking about server-side code. But for the server-side, there are more efficient solutions than just javascript.

imaginenore• 11 years ago

You should optimize your images. These are 6000x4500px 2.4MB PNG images:

http://mrale.ph/talks/goto2015/images/intro-slide.png

http://mrale.ph/talks/goto2015/images/last-slide.png

mraleph8• 11 years ago

Oops, sorry for that. Forgot to shrink them before publishing. Will fix as soon as I get to a place with a stable internet connection - traveling right now.

upd. done, scaled down to 1024x768

gnrlbzik2• 11 years ago

This was a great talk and funny one at that : ) Great perspective on what to really be concerned with. There were other great talks at #GOTOChgo but this one is one of the best ones : )

xenofiend• 11 years ago

How is the acronym LICM pronounced?

mraleph8• 11 years ago

I always pronounce it as /līkm/, but I am not a native speaker so I can't be sure about correct pronunciation.

Benchmarking JS

Related Stories