A First Look at the JIT

https://blog.erlang.org/a-first-look-at-the-jit/

by lelf • 6 years ago

214 62 6 years ago

blog.erlang.org

kzrdude2• 6 years ago

With this approach, it shouldn't be too far off to experiment with ahead of time compilation?

whizzter3• 6 years ago

I think there has existed Erlang to C compilers in the past and there are new one popping up every now and then. In general though AOT is in many ways considered a dead end for many unless you have a certain application for it (even if that was my uni thesis subject).

You can look back all the way to a 1995 paper (1) (with people that then worked on early Java JIT's) that did a comparison of JIT vs AOT compilation for Self, while good AOT is kind of feasible in many ways there is always situations in languages not initially designed for AOT where a JIT won't be forced to go with conservative estimates that hampers performance.

(1) https://www.cs.ucsb.edu/research/tech-reports/1995-04

jecel• 6 years ago

The 1995 paper compares Self 1 and 2 which was a JIT which was based on type inference with Self 3 and 4 which was an adaptive compiler (where Java's Hotspot technology came from) which was able to use type feedback thanks to the second compiler having access to the results of running code from the first compiler.

So AOT compilation was not tested in that paper.

bcardarella• 6 years ago

The Lumen project (https://github.com/lumen/lumen) is building a BEAM bytecode compatible AOT compiler. Its primary compilation target is WASM/WASI. Still under very heavy development. (and some things waiting on updates to WASM spec to land)

whizzter3• 6 years ago

My take on it is that I like most of the design choices they've made (Been tinkering on a similar dynamic runtime before).

With a dynamic language like Erlang (as with JS,Lua,etc) the main benefits won't come from better register allocation,DCE,etc that is the primary reason one would pick LLVM but rather just getting rid of the interpreter overhead (and later the oppurtunity to do some higher level type dependent optimizations that will have a higher impact than register allocation and instruction selection).

Type dependant stuff why LLVM is ill suited IMHO and why you only see LLVM being mentioned as the highest optimizataion level by for example JavaScriptCore(Safari) when the runtime hasn't deoptimized code and is pretty certain of the types that will be used.

Also they mentioned the complexity of maintaining interpreter/JITed jumps and I'm not surprised since i remember reading some paper about one of their earlier attempts and they were maintaining dual stacks with quite complex cross-boundary jumps.

Some might mention that V8 moved away from the always-JIT paradigm to save on memory and startup time but since Erlang is mostly used in servers i think they might see this as a good compromise.

chrisseaton9• 6 years ago

that will have a higher impact than register allocation and instruction selection

The most powerful JIT compiler that I have worked with - Graal - generates code that often on the face of it seems puzzlingly inefficient in terms of registers allocated and instructions selected. Turns out maybe it's another thing that's not as important as all the text books say?

The important bit is removing object allocating, removing boxing, deep inlining, and specialisation... when you've done all that work the exact registers and instructions don't seem to really make a huge difference.

Rochus12• 6 years ago

Are there any papers or articles about the mentioned findings?

MaxBarraclough2• 6 years ago

Could that be due to the cleverness of modern heavyweight CPUs, with techniques like register renaming? Would things change if you used less sophisticated processors?

KMag• 6 years ago

Even with inlining and escape analysis to remove unnecessary boxing, unboxing, and heap allocations, most Java programs still do a lot of pointer chasing. All of that (often poor locality) memory traffic tends to hide the overhead of unnecessary register moves and even some unnecessary register spills.

higerordermap• 6 years ago

How fast is code generated by graal?

In languages like Java w/o value types & virtual methods everywhere, interprocedural analysis is very important. Even otherwise I believe interprocedural optimizations are more important than small register fiddling or cute integer arithmetic tricks. But when you reach a dead-end in optimized code, all these things matter.

Sure, it may not matter for Erlang / Elixir on server.

nickjj3• 6 years ago

Erlang is mostly used in servers i think they might see this as a good compromise.

I'm not sure what the differences will be but start up time is still a very big deal for web servers.

For example I can start up a non-Dockerized Python / gunicorn app with 8,000 lines of code and 50+ package dependencies in about 400ms on a $5 a month DigitalOcean server without doing any type of optimizations.

For someone who just wants to deploy a project to 1 server that means there's only 400ms of down time when I restart my server.

If a runtime takes let's say 10 seconds to start up, that could be enough of a deterrent to not use it if you know you're going down the route of a single server deploy.

cmrdporcupine• 6 years ago

I don't think we're talking "start time" so much as "warm up time"; if I understand these things correctly it's likely the VM would start almost immediately, but would take a little bit of time for hot code paths to JIT and become highly performant. I don't think that would be much of a concern in your example.

dnautics3• 6 years ago

For someone who just wants to deploy a project to 1 server that means there's only 400ms of down time when I restart my server.

Erlang already gives you robust primitives to do things like blue-green deploys, and even graceful transfer of server state even across versioning changes. If downtime between releases is an something you care about, you should use those, and it's likely that your downtime can be in the microseconds range regardless of the vm startup latency.

kannanvijayan• 6 years ago

Some might mention that V8 moved away from the always-JIT

They did, but their always-JIT was a bit heavyweight. They generated full method-jit code on first invocation at first. This was a massive improvement over the existing state of the art, but it was also relatively early in the whole development of the "fast JS" ecosystem we live in now.

The _size_ of JS grew, and the size of functions grew, and the complexity of logic grew. The amount of cold and run-once code grew.

That issue was brought into focus by the growth of the webapp space and the size of the payloads.

In this case (erlang folks) - they're going about the whole thing in a very good way. Their "jit-everything" is actually "jit one instruction at a time", which is _amazingly_ perceptive of the challenges people in other JIT teams have faced (e.g. JS JIT teams). What they're doing is tightly scoped, easy to bootstrap and test with fallbacks to VM-calls into the VM for slowpaths or complicated stuff.

That's a solid base you can slowly layer with higher tiers later if it matters. They're keeping their abstraction layers strong by having a well-specified bytecode system, and hopefully they will strive to keep it relatively independent of the runtime, and avoid leakages of runtime semantics into instructions.

I was personally very impressed by their description of their approach and motivation behind each decision.

Rochus12• 6 years ago

Interesting. Here is also a podcast about the topic: https://thinkingelixir.com/podcast-episodes/017-jit-compiler...

Had a brief look at asmjit; as it seems it only supports x86 and x86_64 and is not really an abstraction (i.e. a platform independent IR). I will try to find out why they didn't use e.g. LLVM or sljit.

EDIT: according to this article (https://www.erlang-solutions.com/blog/performance-testing-th...) the speed-up factor caused by the JIT is about 1.3 to 2.3 (as a comparison the speed-up between the PUC Lua 5.1 interpreter and LuaJIT 2.0 is about factor 15 in geometric mean over all benchmarks, see http://luajit.org/performance_x86.html).

alberth2• 6 years ago

Re: “it seems it only support x86 and x86_64”

Probably because it uses DynASM and I believe only those platforms are supported.

https://luajit.org/dynasm.html

Rochus12• 6 years ago

Probably because it uses DynASM

I had a look at https://github.com/asmjit/asmjit/tree/master/src/asmjit and didn't find any indication that DynASM is used. Anyway, DynASM and LuaJIT are available for many different architectures, not only x86 and x86_64.

tachyonbeam• 6 years ago

LLVM is much too heavyweight for a JIT. It's slow at generating code, and makes it difficult to implement common JIT optimizations such as inline caches, which rely on code-patching.

Rochus12• 6 years ago

Yes, LLVM is huge, but there are a couple of interesting alternatives such as sljit, see e.g. https://github.com/wdv4758h/awesome-jit

Even LuaJIT can be used as a backend for other languages than Lua, see e.g. https://github.com/rochus-keller/oberon/.

saurik2• 6 years ago

(Which is, of course, ironic given that the entire original goal of LLVM was only to build a JIT; it was too slow, but made an OK enough compiler backend ;P.)

dnautics3• 6 years ago

Also keep in mind that the lua speedup is in interpreted vs compiled, and erlang is already a compiled language that had a fairly performant intermediate bytecode.

Rochus12• 6 years ago

But the bytecode is interpreted, isn't it? The PUC Lua VM is a bytecode interpreter as well.

Asm2D• 6 years ago

There is an aarch64 branch in AsmJit project that provides an experimental AArch64 backend. It's pretty complete.

Having a platform independent IR in AsmJit is not planned. AsmJit is more about control and ISA completeness. IR could of course be implemented on top of AsmJit, but it seems nobody really needed that so far - users that need complex IR can use LLVM and users that want something really simple can use SLJIT. AsmJit is just in a different category and it has a lot of useful features considering its size.

moonchild• 6 years ago

Recommend changing the title to '...Erlang's JIT' or similar (though perhaps the 'erlang.org' domain provides enough context?)

haberman• 6 years ago

Data may only be kept (passed) in BEAM registers between instructions.
This may seem silly, aren’t machine registers faster?
Yes, but in practice not by much and it would make things more complicated.

My understanding is that the latency of an L1 cache load is 5 cycles on recent Intel processors.

In tight code that is a really substantial overhead. I get that register allocation is complicated, and I totally see the benefit of simplicity, but it's hard not to think that this will significantly limit performance.

didibus• 6 years ago

It does limit performance, but not in a relevant way for the type of applications Beam is designed for.

CJefferson• 6 years ago

Often, if your function is that small, inlining it is a better idea anyway?

mathw• 6 years ago

Don't lose sight of two things:

Erlang is designed for massive, distributed, highly reliable network systems. Therefore, it is likely that a lot of Erlang code does a lot of waiting around for the network rather than needing every available CPU cycle.

This is the first time they're releasing a JIT. They'll deliver their performance gains, and if they want to and it seems feasible they can come back and do further work in the future.

But you have to put work in where it's valuable, and somehow I doubt RabbitMQ (for example) is going to care about this being missed this time around.

a1369209993• 6 years ago

If the address of the BEAM register file (rbx, apparently) stays constant, and the last write to that register was >5 cycles ago, the load can be speculated 5 cycles in advance; if the last write was <5 cycles ago, my understanding is that that gets handled by store-to-load forwarding with a latency of ~0.

CalChris• 6 years ago

Skylake Client is "4 cycles for fastest load-to-use (simple pointer accesses) 5 cycles for complex addresses"

https://en.wikichip.org/wiki/intel/microarchitectures/skylak...

crazypython• 6 years ago

How is this a JIT? Everything is compiled to machine code. There's no advanced type specialization or lazy deoptimization. It might as well be an in-memory, lazy, AOT compiler.

rurban• 6 years ago

A simple method jit is also a good jit, even without escape analysis and advanced optimizations. It avoids the op branching and enables code caching.

yxhuvud• 6 years ago

It might as well be an in-memory, lazy

Yeah, that is a jit. Perhaps not a very fancy one, the same way ARC is a very unfancy GC, but it is still a jit.

chrisseaton9• 6 years ago

Everything is compiled to machine code.

Yes... but it does that just-in-time, which is what JIT stands for.

There's no advanced type specialization or lazy deoptimization.

I don't think those things are required for it to be a JIT are they? I'd say it's a static JIT, rather than a dynamic, profiling, and specialising JIT, but both are JITs.

It might as well be an in-memory, lazy, AOT compiler.

Well yeah... that describes a JIT.

waynesonfire• 6 years ago

really enjoying this series. excellent job explaining what would be a complicated topic for newcomers.

PeCaN• 6 years ago

I sort of wonder if this approach to JITing is worth it over just writing a faster interpreter. This is basically like what V8's baseline JIT used to be and they switched to an interpreter without that much of a performance hit (and there's still a lot of potential optimizations for their interpreter). LuaJIT 1's compiler was similar, although somewhat more elaborate, and yet still routinely beaten by LuaJIT 2's interpreter (to be fair LuaJIT 2's interpreter is an insane feat of engineering).

htgb• 6 years ago

They detail why they chose the current path rather than continuing improvements of the interpreter in the previous post [1]. See the last few paragraphs for a summary.

[1] http://blog.erlang.org/a-closer-look-at-the-interpreter/

A First Look at the JIT

Related Stories