JVM implementation challenges: Why the future is hard but worth it [pdf]

http://cr.openjdk.java.net/~jrose/pres/201502-JVMChallenges.pdf

by pron • 11 years ago

98 74 11 years ago

cr.openjdk.java.net

nusbit• 11 years ago

Is there a video?

oldmanjay• 11 years ago

Oh please give me continuations in the VM and I will take back every angry thought I've ever directed at Oracle.

haddr• 11 years ago

Lightweight threads (fibers) looks especially interesting to me. I wonder if there is some library that already tries to ger around threads with something more lightweight...

norswap• 11 years ago

Assuming this is not rhetorical; yes there is: http://www.paralleluniverse.co/quasar/

tormeh3• 11 years ago

http://akka.io/

eranation2• 11 years ago

I love Akka, but I thought their implementation relies on regular JVM threads at the moment, isn't it? (I had a wishful thinking at first that they are using user-threads at first, but they are not like quasar or goroutines in any way from what I read, Actors might reuse threads and seem like a "lighter thread" but they are still using plain old JVM threads, for now at least)

x0x04• 11 years ago

fj / jsr166 is in 7

sample code: http://docs.oracle.com/javase/tutorial/essential/concurrency...

rdtsc• 11 years ago

An interesting one is

http://docs.paralleluniverse.co/quasar/

They claim to have "true lightweight" threads. (I am sure pron if he is a around can jump in and expand on it).

It takes a lot from Erlang even pattern matching.

pron7• 11 years ago

Well, the pattern matching is only in the Clojure API, but yeah, these are true fibers based on continuations implemented with bytecode instrumentation, scheduled by a work stealing scheduler (or any other scheduler of your choice). It lets you write simple blocking code with all its familiarity and advantages (exceptions, thread-local data) but enjoy the performance of async code.

The downside is that libraries have to be integrated in order to block gracefully when called on fibers, but we already have a long and growing list of integration modules for popular libraries.

justinsb• 11 years ago

I would love to see threads based on this work (OS support for lightweight threads): http://www.linuxplumbersconf.org/2013/ocw//system/presentati...

It seems this would move most of the heavy lifting into the kernel (and out of the already-very-complicated JVM). Anyone know what happened to that work?

pron7• 11 years ago

AFAIK, Google uses it (to some extent, at least), but the solution suffers some issues that make it less than ideal. While it relinquishes scheduling to the application, stack management remains the kernel responsibility, so while the task-switching overhead is greatly reduced, the threads are still not quite lightweight. The advantages to the approach is that it automatically crosses "language barriers", say, between the JVM and native code. This is very important for Google, who use C++ libraries as common code in their polyglot projects, but maybe not so important in other shops.

x0x04• 11 years ago

There's some exciting stuff in the presentation. In particular: next gen threading (fibers/warps, fj), plus attention to java's profligate use of memory b/w because of it's reliance on pointers-for-all-the-things. Plus the idea of the jvm helping out languages besides java is also exciting.

That said, it's a little scary to contemplate the power that microsoft, steward of the clr, and oracle, steward of the jvm have over our industry. It's staggering to see the amount of engineering that's gone into bullet-proofing the jvm. What happens when microsoft or oracle decline to continue paying for hundreds of very expensive senior compiler/vm/language engineers?

tormeh3• 11 years ago

I've long wondered what's in it for Oracle and now Microsoft. Microsoft stands to profit from applications written for Windows that are hard to port, but they seem to be abandoning that approach. Oracle seems to have no real upside to maintaining Java. No one is paying for C# and Java directly, so I don't get it.

xxxyy• 11 years ago

Java is the corporate powerhouse, especially for banking. Together with Oracle DB. Owning two out of three crucial components (the third being the hardware) creates a lot of good business opportunities for them, makes it easier to enforce monopoly.

parkovski• 11 years ago

Both companies have a lot of enterprise software that exposes either Java or .NET APIs, and lots of companies pay them for support and troubleshooting, and licensing those other products.

pjmlp7• 11 years ago

Oracle seems to have no real upside to maintaining Java

Since the early 2000 the majority of Oracle GUI tools across their products are written in Java.

Long before acquiring Sun, Oracle did acquire Bea.

Oracle still sells Bea Weblogic JEE container and the J/Rockit JVM, which also has a real time version.

They have their own JSF framework and IDE.

So becoming Java steward is not so strange.

tsotha• 11 years ago

I always believed Sun shot itself in the head with Java. They had this idea getting people invested in a cross platform ecosystem meant new software could be run on Sun hardware.

But what really happened is people who had been locked into Sun with legacy software wrote applications in Java and then switched from Sun to cheaper hardware running Linux or Windows.

gaius• 11 years ago

Oracle seems to have no real upside to maintaining Java.

No, but they have an awful lot of downside from not maintaining it.

Consider the history of IBM, their grand strategy in the early 90s was that Smalltalk would be the enterprise language of the next few decades. The Java juggernaut rode roughshod over IBM. Or Microsoft, they would have loved to have kept Visual Basic and Visual C++ cash cows going forever, Java completely blindsided them. Altho' it may be technically less powerful than Smalltalk or C++, in terms of mindshare (or hype if you prefer) Java is too powerful for Oracle to risk falling into anyone else's hands.

Padding• 11 years ago

No one is paying for C# and Java directly

No one you know.

Suppose some large and wealthy entity has trouble running some money-making legacy application that happens to be running atop the JVM. Who do you think will they call to fix their issues asap?

.. Which is why there's a lot of interest in having the people working on the JVM on your payroll ..

LaSombra• 11 years ago

For Java there's always OpenJDK, which I believe is 100% open source.

wtetzner• 11 years ago

Well, there's OpenJDK and now the CLR is open source, so companies that are heavily invested in the platforms can pay for people to work on them.

As for the languages and compilers, I'm less worried. There are already plenty of open source languages targeting these platforms, many of them better than Java/C#.

pjmlp7• 11 years ago

What happens when microsoft or oracle decline to continue paying for hundreds of very expensive senior compiler/vm/language engineers?

People will move away for something else, while a few will keep on maintaining them.

I remember when the answer for any database related problem was Clipper. Now it is gone.

I remember when no one would question the use of Turbo Pascal for both system programming as application programming.

I remember when using a spreadsheet meant Lotus 1-2-3.

I remember ....

aragot• 11 years ago

Sorry I "arrived late at the party", was Turbo Pascal ever used beyond business applications? I've always thought it was a cool programming language but that C was the massive leader.

pron7• 11 years ago

There are now many large companies participating in OpenJDK. Of them, IBM, AMD and Intel are contributing code (Google, sadly, not so much).

tomp• 11 years ago

On slide 16, which talks about cache, there is this:

Rule #1: Cache lines should contain 50% of each bit (1/0) – E.g., if cache lines are 75% zeroes, your D$ size is effectively halved

Can anyone explain this?

x0x04• 11 years ago

a cache line is the unit of read from ram, and is at a premium

At least for data, a common win is to compress the data in ram and decompress once it is in-core; this is often essentially free as many machine learning algorithms are bandwidth starved but have plenty of compute available. Suppose you are eg storing small integer counts in ints; if it's a java int you are using 4B to store 1B, while if it's a java.lang.Integer it costs 16 bytes plus most likely an 8B pointer.

Another way to consider this is if you are using 8B pointers, you waste a lot of that as constant zeros -- 1TB is 2^40, so even on a 1TB machine 3B/24b are wasted. Particularly with (all? at least that I know of) jvms that have 8B alignment, another 4 bits are wasted. This is how the CompressedOops hack words to access 32G ram w/ 32b pointers.

bluecalm• 11 years ago

>At least for data, a common win is to compress the data in ram and decompress once it is in-core

I have problems imagining how it could be done. Would you mind elaborating and/or sharing some examples ?

antoaravinth2• 11 years ago

I didn't get them fully. Sorry for novice questions, but wish to understand your statement completely.

Suppose you are eg storing small integer counts in ints; if it's a java int you are using 4B to store 1B, while if it's a java.lang.Integer it costs 16 bytes plus most likely an 8B pointer.

This wastage does happens only in reference types? Tomorrow say if value types are been created, then these sort of issue will get resolved?

Another way to consider this is if you are using 8B pointers, you waste a lot of that as constant zeros

Why of constant zeros? And in the line your saying "-- 1TB is 2^40", your referring 1TB of cache line?

jbn• 11 years ago

if your cache line is 75% zeroes, then half of it respects the objective of "should contain 50% of each bit", and the other half is all zeroes, which is a waste (according to this objective)... in other words, only half the cache is 'correctly' used. This is just another way of saying that your D$ size is effectively halved.

pjmlp7• 11 years ago

Great to see AOT compilation coming to the reference JVM instead of relying on third parties.

Also the desire to go meta-circular and reduce even more the amount of C++ code.

Most likely based on the Graal/SubstrateVM work.

Finally JNI being replaced by something developer friendly.

vardump8• 11 years ago

Cache, cache and cache. Three of the most important things Java gets wrong today. Mostly because Java doesn't have object arrays at all. Only primitive types can be array elements. So you get array of references to objects instead of array of objects.

There are only 512 L1D cache lines in a CPU core. After just 512 references to memory areas covered by different cache lines, L1D entries will start to drop. Or earlier. In Java this happens very easily. Like when just iterating a single, not very large, data structure.

Everything else (except maybe SIMD/GPU related items for compute purposes) are not anywhere near as important as fixing Java's current problems with CPU caches.

pjmlp7• 11 years ago

Back when Java still wasn't known, we had Oberon(-2), Component Pascal and Modula-3.

All system programming languages with GC, value types and AOT compilers (some implementations of Oberon did have JITs).

Apparently 20 years detour was needed to go back to those features.

seanmcdirmid2• 11 years ago

The CLR has had value types from the onset.

mike_hearn2• 11 years ago

Apparently 20 years detour was needed to go back to those features.

I think this is explainable by Java's history. When it was created, the design docs said Java was a simple, interpreted language with distributed objects and the target hardware was set top boxes. When Java unexpectedly took off, I suspect Sun weren't quite sure which ingredients were key to that success but fixated on "simple". They unlocked a huge market that was unserved at the time: companies that needed a much simpler C++. So that wasn't unreasonable. They then spent many years trying to handle customer requests for ever more runtime speed, whilst simultaneously not making Java as complex as C++. Hence the focus on super whizzy optimisations inside HotSpot ... many of which are just undoing performance hits introduced by the simplicity of the Java language.

Value types increase the complexity of the language, no doubt about it. Heck, even having a difference between int and Integer increases complexity. When value types come out, I expect much rejoicing from people who need more runtime speed, and a whole lot of confused questions on Stack Overflow from people who aren't totally sure about the difference between a value type and a regular class.

I also expect to come across codebases where classes were turned into value types more or less at random, or where the style code forbids value types entirely, etc. Java sort of has this problem already where weaker programmers who are using threads just slap synchronized keywords on every method until the crashes stop. Then HotSpot goes in and tries to figure out if the locks are actually needed or not and removes them at runtime.

Don't get me wrong. I'm A++++ in favour of having value types on the JVM and in Java. But increasing complexity of the language is absolutely not cost free, especially not given Java's value proposition to big corporates.

spullara• 11 years ago

This is a focus for Java 10 — value types. John Rose is actually one of the guys working on it. He mentions this several times in the presentation as a key issue to solve in the JVM.

https://www.google.com/search?client=safari&rls=en&q=value+t...

kristianp• 11 years ago

This reminds me of something I'm curious about: why have L1 caches have been the same size for quite a few generations of Intel Core processors?

vardump8• 11 years ago

Currently Intel's L1D (level 1 data) cache is 512 lines with 64 bytes each, 32 kB. Been that way for a pretty long time.

L1D latency with a pointer is mostly 4 cycles. Not sure, but I think having 1024 entries would increase that to 5 cycles.

Increasing cache line size from 64 bytes to 128 bytes would require more memory (DDRx SDRAM) bandwidth, because CPU needs to always fill a whole 64 byte - 128 byte with this change - cache line on a memory load. Even if you want just one byte. Wasted bandwidth for non-streaming workloads would be increased significantly. This change would force also L2 to have 128 byte cache lines, etc. LFBs and other buffers would of course also need to accommodate this.

Skylake will be able to process 64 bytes (512 bits) with a single SIMD instruction, with AVX-512. Perhaps Intel will need to soon increase cache line size, if 1024 bits wide SIMD instructions are desired.

SIMD gather instructions might be a hint that in the future memory loads will function in a different fashion. Maybe cache lines won't anymore be the smallest amount of data a CPU memory subsystem can deal with?

wmf2• 11 years ago

Because that size is optimal, regardless of transistor size. Making the cache larger makes it slower (because of the speed of light AFAIK) and more power-hungry which reduces overall performance.

jjoonathan3• 11 years ago

This reminds me of something I'm curious about: why aren't large SRAM arrays displaying DRAM? I know 6Ts are larger than 1T+1C, but even if SRAM were 6x as expensive (which ought to be a severe upper bound) that would put it at $50/GB which sounds like it could easily be worthwhile for certain workloads -- not unlike SSDs a few years ago. The latency advantage would be gigantic, especially if you could crowd the chips near the CPU.

Is SRAM comparatively power hungry or something?

im3w1l• 11 years ago

It'd go against OOP, but you could take all the member variables and put in separate arrays.

E.g instead of an array of 3-vectors, you'd have an x-array, a y-array and a z-array.

pkolaczk• 11 years ago

Still if you need all three together, that's potential 3-cache misses instead of 1.

jermo• 11 years ago

ObjectLayout [1] should help with that but its not in standard Java.

[1] http://objectlayout.org/

harkyns_castle• 11 years ago

Some of the talks [1] on the LMAX Disruptor [2] were interesting with regard to how it interacted with the various caches and how important it could be.

1. https://www.youtube.com/watch?v=DCdGlxBbKU4 2. https://lmax-exchange.github.io/disruptor/

JVM implementation challenges: Why the future is hard but worth it [pdf]

Related Stories