Java’s new garbage collector promises low pause times on multi-terabyte heaps

https://www.opsian.com/blog/javas-new-zgc-is-very-exciting/

by ScottWRobinson • 8 years ago

372 233 8 years ago

opsian.com

lamby• 8 years ago

Pointer colouring

(Is this the same as pointer tagging?)

rlmw• 8 years ago

I would say that pointer colouring is a type of pointer tagging. AFAIK the colouring in ZGC represents different states. So the memory page mappings rely on the pointer having one colour (1 of the 4 bits set) whereas pointer tagging is the term for the general concept of encoding information in pointers.

nicktelford2• 8 years ago

GC’s SPECjbb 2015 throughput numbers are roughly comparable with the Parallel GC (which optimises for throughput) but with an average pause time of 1ms and a max of 4ms. This is in contrast to G1 and Parallel who had average pause times in excess of 200ms.

Not bad! Looking forward to seeing how this performs with a diverse range of workloads as it matures.

earenndil3• 8 years ago

I wonder if that would make java more suitable to game development since pauses need to be <8ms, ideally.

mysterydip• 8 years ago

For a subset of developers, definitely. Overall I don't think it would lead to significantly more adoption, as the main draw (write once deploy everywhere) is already accomplished by engines like Unity.

pjmlp21• 8 years ago

Java is already suitable for game development, specially with LibGDX.

Not everyone needs to try to write the next Crisis in it.

It certainly has higher performace than all those HTML 5 and Flash games.

The biggest problem is that jMonkey is the only RAD tooling available and no match against any of engines that make use of C#.

nicktelford2• 8 years ago

I think a lack of value types also hurts Java significantly here. Though I believe they're also on the roadmap.

HohPum1l2• 8 years ago

On smaller heaps it was already possible to get 5-10ms pauses with the (now deprecated) CMS collector. Unlike Z it was a non-compacting collector though and had some ugly worst-case behaviors.

phakding4• 8 years ago

I worked for a company where we had a 100-150 microseconds order latency. We had to tune jvm to extremes. But it is very doable to optimize Java gc for low gc pause times.

VikingCoder2• 8 years ago

Hrm, I would think that if you were developing a game in a GC environment, where GC timing turned out to be a problem, you would get rid of dynamic memory allocation, and try to do everything much more like embedded system programming with fixed memory blocks...?

I suppose at the complexity of modern games that's simply not possible anymore...

jblow4• 8 years ago

8ms? Where do you get that number from?

If your player has a 144Hz monitor then your pause time, rounded to the nearest millisecond, has to be 0ms.

waterbear• 8 years ago

Isn’t Java a pay-to-play runtime now? I thought Oracle changed the licensing model after Java 8?

koolba• 8 years ago

No it’s still free, they (Oracle) are just not supplying the free JVMs. There are plenty of alternatives for OpenJDK builds that are FOSS.

antonvs2• 8 years ago

Oracle still supplies free JVMs. The change was that to get updates for older versions of the JVM, you need to pay.

antonvs2• 8 years ago

No. The change was that you need to pay to get updates for older versions of the runtime.

aseipp2• 8 years ago

What they did was change the update model for paid versions of the standard Oracle JDK -- you must have a commercial license with Oracle in order to receive security and bugfix updates for copies of the Oracle JDK.

OpenJDK, on the other hand, was unaffected by this change; things like security mitigations/fixes land in OpenJDK, and make their way to other versions (like older JDK versions, or Oracle enterprise copies) from there. So, you can go download Zulu or OpenJDK binaries from your distributor (such as your Linux distribution) and you'll be fine (assuming they update promptly).

There are also feature distinctions between the Oracle JDK and OpenJDK, but they have been getting smaller over time -- AppCDS and Flight Recorder, which were Oracle-customer-only features, for example, are now in OpenJDK proper (FR will come in JDK 11), and many features such as ZGC (and originally G1, too, I think) were developed in the open and went into OpenJDK directly, right from the start.

If you're just running a bog standard Linux/BSD system with some Java software on it, you're almost certainly OpenJDK already anyway, and your distribution maintainers handle security updates for you.

dig1• 8 years ago

That is for Java SE8. Not affected for later releases. Beside, ZGC is part of OpenJDK, which is under GPL.

DannyB2• 8 years ago

Get your JVM from Oracle, Azul, Adopt Open JDK, Red Hat.

You can either:

1. Update it every six months, for free, if you want to keep current with security updates.

2. Pay for updates on versions older than six months.

3. Not pay, not upgrade, and live with unsupported versions.

Adopt Open JDK has indicated their intention to back port patches to older versions for several years past their expiration for free.

Twirrim• 8 years ago

ZGC is going to be in OpenJDK, the fully open source, reference implementation, of Java: http://openjdk.java.net/projects/zgc/

All of the proprietary bits in Oracle's Java implementation are also being fully open sourced: https://www.infoq.com/news/2017/10/javaone-opening

matt_heimer• 8 years ago

http://www.oracle.com/technetwork/java/javase/eol-135779.htm..., The Oracle JDK will cost for production deployments but free for development beginning with Java SE 11. OpenJDK will be interchangeable with the Oracle JDK and Oracle will provide OpenJDK builds for free.

_old_dude_• 8 years ago

No, Stephen Colebourne has recently written a blog post [1] on that subject.

[1] http://blog.joda.org/2018/08/java-is-still-available-at-zero...

M_Bakhtiari4• 8 years ago

I'm glad they didn't buy into the refcount meme that's so inexplicably popular these days.

ahamez• 8 years ago

Could you elaborate? Why would refcount be a bad idea ? (genuine question)

M_Bakhtiari4• 8 years ago

Load-increment-store is slow. Load-decrement-branch-if-zero is slow. And you still need a tracing collector for cycles. So it's slow and still nondeterministic.

pjmlp21• 8 years ago

Many think that refcount isn't a GC algorithm and that it is faster than GC.

Well, refcounting is the basic way of implementing GC and is listed as GC algorithm in any serious CS book about GC algorithms like "The Garbage Collection Handbook".

What many refer as GC is actually the GC algorithms that fall under the umbrella of tracing GC.

Then refcounting is only faster than GC in very constrained scenarios:

- no sharing across threads, otherwise locking is required for refcount updates

- no deep datastructures, otherwise the call stack of cascading deletions will produce a stop similar to tracing GC

- implementations need to be clever about nested deletions when refcount reaches 0, otherwise a stack overflow might happen

- cyclic datastructures need programmer help to break cycles via weak dependencies or are just not allowed

This is only relevant for naive refcount implementations, there are many optimisations, which endup turning a refcounting implementation into a tracing GC in disguise.

Also just because a language uses a tracing GC algorithm, it does not prevent the existence of value types, or manual memory allocation for hot code paths, thus allowing for more productive coding, while offering the tools for memory optimisation when needed.

This is not yet the case with Java, but even here it is part of the language's roadmap to fix this.

jwr• 8 years ago

From the hacker folklore (AI koans, available at http://catb.org/jargon/html/koans.html):

Moon instructs a student

One day a student came to Moon and said: “I understand how to make a better garbage collector. We must keep a reference count of the pointers to each cons.”

Moon patiently told the student the following story:

“One day a student came to Moon and said: ‘I understand how to make a better garbage collector...

ridiculous_fish• 8 years ago

In addition to what others have said, the choice of memory management schemes has big knock-on effects. It influences how much runtime metadata you need to retain (e.g. stack maps), it has lots of implications for the FFI, and may constrain various tricks like unions, tagged pointers, NaN boxing, etc. So narrowly comparing schemes in terms of just allocation throughput won't paint the whole picture.

freecodyx• 8 years ago

i did google search for "refcount meme", and found out your comment as 3rd result, so it's not that popular...

jandrese• 8 years ago

I figured he was talking about Rust, given that it is basically a refcount system but with the added constraint that the refcount can not exceed 1.

It's the only language that is anywhere close to meme status that I know of.

tom_• 8 years ago

If reference counting were going to help, they'd probably have switched to it already by now. I wonder if the poster just is angling for an opportunity to embark on a rant about one of Swift/Objective-C/ARC, Python, Rust, C++ - or perhaps all of them?

JeremyBanks2• 8 years ago

I did a search for "freecodyx user", and found no results, so I guess you don't exist.

hinkley• 8 years ago

A lot of the early Java press about their use of GC involved decrying the state of other interpreted languages that were still stuck on reference counting, and the problem of circular references.

codr42• 8 years ago

"It is time to unmask the computing community as a Secret Society for the Creation and Preservation of Artificial Complexity." ~Dijkstra

It's very impressive work, no doubt. And as long as no one asks me to debug anything running on top, I don't have any issues with it.

At least with straight reference counting I know what's going on under the hood, that's worth a lot. Sharing pointers across threads is tricky business, I can't really see what that has to do with anything. And it's quite likely that it will run faster, simply because it's much less complex.

DonHopkins• 8 years ago

I've always thought the Monty Python Sketch about The Royal Society for Putting Things on top of Other Things was a perfect metaphor for modern "full stack" software development.

https://www.youtube.com/watch?v=1f-kfRREA8M

And the Ministry of Silly Walks is the perfect metaphor for user interface design.

https://www.youtube.com/watch?v=iV2ViNJFZC8

frankmcsherry• 8 years ago

For people who want to fight about tracing vs ref-counting, please read this first, on how the two are dual and most collectors use both:

http://www.cs.virginia.edu/%7Ecs415/reading/bacon-garbage.pd...

obl• 8 years ago

The memory mapping trick they use on x86 to avoid masking only works up to the maximum 48 bits of addressable virtual memory, so there is much less than 22 free bits in that case.

It's also not quite free since it takes up TLB space.

DSMan195276• 8 years ago

Agreed - I'd go farther to say it only works for addresses in the userspace half, so on Linux you lose an extra bit and only get 47 (I don't know how the mappings are setup for Windows/OSX, sorry). It also might be worth pointing out that you now need 16x the page mappings (Since every mapping needs 15 duplicates for the possible flag states) - I don't know how Java does it's memory management but if it does lots of small mappings (Which I'm guessing it does not) then that could be a concern. And like you mentioned with the TLB, it takes up space and it also slows things down a bit since if the flags change the entry in the TLB will no longer be used (since the address is now different) and the MMU will have to walk the page-table again.

In practice, I would wager the performance concerns aren't huge, but that's mostly just a guess. I'd personally be interested in seeing a comparison to masking to see just how much slower masking would be, but obviously it's not like they can just flip a switch to use masking.

HohPum1l2• 8 years ago

It also might be worth pointing out that you now need 16x the page mappings

It doesn't. The article mentions it's only 3 mappings since those bits are colors, not arbitrary combinations of flags.

I don't know how Java does it's memory management but if it does lots of small mappings (Which I'm guessing it does not) then that could be a concern.

openjdk generally uses large contiguous mappings but it may punch holes in the middle of the heap if it's configured to yield back memory to the OS. But applications that dynamically shrink and expand their heaps are not necessarily those that are concerned about the last quantum of page table overhead.

And like you mentioned with the TLB

On the other hand it does support huge pages to mitigate costs of TLB entries.

pitaj• 8 years ago

It also might be worth pointing out that you now need 16x the page mappings (Since every mapping needs 15 duplicates for the possible flag states)

Nope. They went into this in the article:

Since by design only one of remap, mark0 and mark1 can be 1 at any point in time, it’s possible to do this with three mappings. There’s a nice diagram[1] in the ZGC source for this.

[1] http://hg.openjdk.java.net/zgc/zgc/file/59c07aef65ac/src/hot...

That might not be totally current, as it doesn't cover the finalizable flag, but if it works the same, that would only be four mappings. If it works differently, then it would be a maximum of 6 mappings. Not 16.

adrianmonk• 8 years ago

I agree that it definitely can't be completely without cost. But I'll speculate that they may have arranged for the don't-care bits in the pointers to all be 0 when GC is not running and doesn't need the bits. If so, that could mitigate the TLB waste during periods when GC isn't running.

In other words, just because the alternate mappings exist doesn't mean the pointers always have values that actually use them.

joemag• 8 years ago

Many Java applications will gladly take 10%, or even higher, performance hit if that means more predictable GC times. If you own a latency sensitive application, then it's common for tail latencies to be dominated by GC times. Bringing tail latencies closer to median would be a huge win for those app, even if the median itself moved.

gct• 8 years ago

x86 caches uses virtual address for indexing (physical bits for tag), so this'll increase cache pressure too.

titzer2• 8 years ago

It takes more TLB entries, but not physical cache space. Virtual indexing just makes use of the page offset being the same between virtual/physical mappings in order to select the cache set before mmu translation is available.

igravious• 8 years ago

4Tb ought to be enough for anybody.

felipemnoa2• 8 years ago

Reminds me of the quote "640K ought to be enough for anybody."

(I'm guessing you were think of the same [Which apparently is just apocryphal])

jopsen• 8 years ago

I can't imagine many use cases where you wouldn't want to use multiple processes before you hit 4TB. But yeah, I'm sure there are some...

M_Bakhtiari4• 8 years ago

Why would you want to introduce the extra complexity of IPC just because your GC sucks and can't handle the heap size that you need?

Besides, processes are a ham fisted way to get around the PDP-11's limited address space and shouldn't be a thing in 2018 anyway.

cridenour• 8 years ago

All I could think during reading - is this going to affect my Minecraft server parameters?

stuff4ben• 8 years ago

Interested to know what workloads are using multi-TB heap sizes? Largest I've used has only been in the 48-64GB size and even then it seemed like a waste.

pjmlp21• 8 years ago

I have only heard about it in the context of Fintech, where they keep all transaction data around.

At least they seem to be Azul´s biggest customer base.

kjeetgill5• 8 years ago

1.2 TB Xmx (Java heap) here, reporting in from LinkedIn not fintech.

Basically, graphs are hard to shard ... what if we didn't bother? You can query an in memory graph pretty quickly out into the 2nd or 3rd degree.

Even G1 (untuned) only pauses for 250ms once every few minuets. It's not perfect but good enough to ignore.

hermitdev5• 8 years ago

250ms is atrocious for finance. Need a small handful of microseconds to not be noticed.

amelius2• 8 years ago

Nice. But unfortunately, it's owned by Oracle. Use it, and you'll be constantly worrying what their lawyers are up to next.

kevinherron• 8 years ago

ZGC is part of OpenJDK and licensed under GPL.

amelius2• 8 years ago

Doesn't matter. Our legal system is such that Oracle can still burn you down, even if you are right.

humbleMouse5• 8 years ago

Openjdk8 or get out! Seriously though I am concerned about the future of Java. I am keeping everything openjdk8 personally.

guipsp• 8 years ago

Why?

redsymbol• 8 years ago

OT: I initially mis-read this as "Jay Z's new garbage collector".

Talk about an intriguing headline.

exabrial• 8 years ago

Limited to 4tb heaps

I can't believe the world we're living in, this is incredible. My first computer 512k of ram.

dboreham2• 8 years ago

512 bytes here.

Cyph0n2• 8 years ago

What kind of PC had 512 bytes of RAM?

userbinator• 8 years ago

Java is at least somewhat responsible for the need to get to 4TB in the first place.

felipemnoa2• 8 years ago

Sounds interesting. Could you explain how so?

Tarean2• 8 years ago

There is also an interesting trick added for Shenandoah to make thread synchronization time bounded.

Basically, you don't want to check for gc in each (allocation free) loop iteration. On top of the overhead it makes optimization harder. But if you have 32+ threads then some are probably stuck in loops for a good while which means spinlocks for everyone waiting for the stop-the-world gc.

The trick to avoid this is to add an inner loop that does a fixed number of iterations and then do a gc check between each inner loop https://bugs.openjdk.java.net/browse/JDK-8186027

schmichael• 8 years ago

today AWS will happily rent you an x1e.32xlarge with 128 vCPUs and an incredible 3,904GB of ram.
...
ZGC restricts itself to 4Tb heaps

Isn't a 4TB heap limit short-sighted for a GC intended to last over a decade?

phoe-krk• 8 years ago

4TB uses 42 bits of address space. It leaves you with 22 bits, out of which ZGC uses 4.

If anything, borrow a few more bits, and you should easily be able to address petabytes.

chrisseaton4• 8 years ago

4TB uses 42 bits of address space.

Can't you compress that into less than 42 bits?

foobarbazetc• 8 years ago

We use Shenandoah everywhere. It’s amazingly great.

kjeetgill5• 8 years ago

Is that checked into OpenJDK for the 11 release? Does it have any advantages to ZGC?

aseipp2• 8 years ago

No, it's still a separate fork for Java 8/10/11 as of right now, although some distributions like Red Hat/Fedora do ship with it enabled as a "Technology Preview" on all their enabled JDKs, so you can just get it out of the box. I think Gentoo has it as well.

(If it also means anything, I can confirm that enabling Shenandoah is as simple as taking the corresponding OpenJDK code and replacing the `hotspot` subcomponent with the (compatible) Shenandoah fork'd hotspot component, then just building everything normally.)

ryanobjc• 8 years ago

An interesting fact is that the hotspot team went back on their 20-year conviction that load-barriers reduce performance too much. I guess improvements in CPU and VM designs has changed their minds.

This is a big deal, and a major departure.

I'm hoping this will improve things, this new architecture more closely resembles Azul's C4. Java and large heaps with non-conforming object allocation patterns (medium aged objects are the most troublesome) sucks really badly.

repolfx3• 8 years ago

They do reduce performance a lot. I think what changed is there are now classes of customers who are willing to take a huge performance hit for reliably low pause times or reliably huge heaps. Whereas in the past heaps were smaller and perhaps latency tails were something people cared about less.

ADefenestrator4• 8 years ago

G1 does a lot better here in my (somewhat limited) experience. There's some overhead, and you have to give it some extra space, but the mixed collections are decent at making their way through medium-aged objects that survived to the old generation. Still nowhere close to 5-10ms though, and with a fair bit of memory bandwidth burned copying it through the survivor spaces.

quotemstr• 8 years ago

"For sale: baby shoes, never worn" has nothing on "low pause times on multi-terabyte heaps"

AboutTheWhisles• 8 years ago

Who are these people that are designing systems that have terabytes of heap under a single GC, but still want/expect/need low pause times.

I would think if there were processes that needed consistent latency you would isolate them.

Inityx2• 8 years ago

Probably really old teams that had their scope change over time, with decades of technical debt, whose large controlling entities are too slow-moving to afford any kind of large refactor or rewrite in a more suitable language.

Aweorih• 8 years ago

I think I read somewhere, that the heaps of ecommerce website applications can be very huge, because of the loaded strings. I think that was in the time of the java update (9?) where reduced string size was added

openasocket• 8 years ago

I'm not entirely sure what the point of the multi-mapping is. It says that it means you don't have to do any pointer bitmasking to dereference, but you already have load barriers, which have to at least check to see what phase you are in and possibly do work every time you dereference a pointer, so how much extra latency would doing the bitmask really add? And the problem with the multi-mapping is you will be resolving more virtual addresses, increasing the working set for the TLB. Seems like there could be increased TLB thrashing.

silverlake• 8 years ago

Some platforms like SPARC have built-in hardware support for pointer masking so it’s not an issue but for x86, the ZGC team use a neat multi-mapping trick.

Does anyone know what this instruction on SPARC is?

Java’s new garbage collector promises low pause times on multi-terabyte heaps

Related Stories