GC Tuning Confessions of a Performance Engineer

http://www.slideshare.net/MonicaBeckwith/gc-confessions

by luu • 11 years ago

35 95 11 years ago

phamilton2• 11 years ago

For some reason, GC is on of those areas that I have zero desire to learn sufficiently to tune. That's probably why OP has a successful career as a consultant.

When making tradeoffs in development, a simpler GC with predictable behavior is worth losing a lot of raw performance to me. Thus I find myself drawn to platforms like erlang/elixir or Haskell where GC is isolated.

I suppose the rational approach is build on Hotspot and when it becomes a problem hire a consultant for instant performance gains (assuming low hanging fruit is in abundance).

pron17• 11 years ago

In 80% of the cases, simple GC ergonomics (i.e. let HotSpot figure out the tuning) are more than ok, and in 95% of the cases, trivial tuning is enough.

The thing with Erlang is that the GC doesn't work on any shared memory data structure (like ETS), so you pretty much have to delegate any shared data to an out-of-process database, even in simple cases that are easily addressed with ConcurrentHashMap, ConcurrentSkipListMap/Set, or CopyOnWriteArrayList/Set on the JVM.

phamilton2• 11 years ago

I generally use an Agent in Elixir for anything shared and there's very little cognitive overhead (though perf does take a hit). Agent.cast/2 is helpful for low latency modifications (at the cost of losing back pressure).

As stated, it's a tradeoff.

duaneb4• 11 years ago

Shared memory data structures are rarely necessary. When one is, I would be willing to go the extra mile to facilitate it. Seems like a reasonable tradeoff.

rdtsc2• 11 years ago

Good point, but ETS is still managed and owned by a process. So it still can be cleaned up taken care of. Moreover at the API level ETS even behaves as if there is a process that manages data (even though implementation-wise it is all done in C).

Also in Erlang large binaries are not copied but shared as well.

And whether large shared data-structures are needed is dependent on the application. I imagine since you've done work with spatial relationships, in that case for anything interesting to happen data has to be indexed, shared, queried in one place. But say building a large concurrent chat or messaging application with many but small state machine actor that might have lots of peer to peer interaction, then shared data is not as crucial.

rdtsc2• 11 years ago

Yeah that is why I like the BEAM VM as well. It is a really awesome piece of engineering build for massive concurrency and fault tolerance. It is nice to see the whole Elixir ecosystem take advantage of it.

That model of concurrency is important to reduce cognitive load on the programmer. Having concurrent units and mutable state isolated makes it much easier to understand what is going on, rather than having some large class or shared memory state modified from many code paths concurrently (maybe via callbacks, threads, signals, etc.)

jeffcoat• 11 years ago

The slides are interesting enough that I went looking to see if the actual talk that goes with them was online anywhere; found it at

http://chariotsolutions.com/screencast/philly-ete-2015-7-mon...

banachtarski13• 11 years ago

Having worked with all sorts of GCs in the past, I basically stopped using them altogether some years ago in favor of manual memory management. I have to say, it's been liberating and so much easier to write performant code since I know what memory I need instead of relying on the computer to guess (and giving the computer hints as to how to guess). I have no doubt that GC research and systems is making good forward progress, but I can't imagine myself using them again.

pron17• 11 years ago

I love GCs. They give you better memory throughput (in exchange for more footprint and higher latency, although latency can be made rather low), and they let you build and use very scalable concurrent data structures. On large machines with lots of cores and lots of RAM, they let you work with large, in-memory data sets very efficiently.

duaneb4• 11 years ago

As with everything in engineering, this is a tradeoff. Not all access patterns work well with mark-and-sweep (or equivalent full traversal patterns). In particular, without needing to release or allocate memory, a GC pass wastes cycles. Furthermore, data structures must be compatible with the GC: traversal is proportional to the number of pointers (rooted or dangling) in the heap and stack at the time of running.

Generational garbage collection helps this, but for large numbers of pointers, you're going to see performance decrease unless at some point you move the pointers out of collection contention--think an allocation pool to reduce the depth of traversal early. I believe google just released a library to help with this, though the semantics are specific enough you could easily make your problems much worse.

GC is never a flat win.

banachtarski13• 11 years ago

You will never beat a tuned system without GC with a GC system because of all the instructions necessary to traverse references and such. Having lots of cores and lots of RAM just means you can eat the cost more easily. It just means you can afford to let the VM do the work instead of the programmer.

Also, no matter how much RAM you have, cache sizes are more or less the same, and cache line misses hurt.

Dewie32• 11 years ago

Not to mention that you get to hire performance engineers.

People may have overestimated their utility (that they are good for less things then commonly thought). Maybe not to the degree that the stereotypical C/++ would believe, but still an overestimation.

Maybe we just need to use more abstractions in the middle between manual and fully automatic memory management, like region-based memory management (just an example).

bd_at_rivenhill• 11 years ago

I think we will start to see less use of GCs in mobile going forward, especially now that Apple has deprecated GC in favor of automatic reference counting. The issues caused by the interaction between the memory hierarchy and the fragmentation that is naturally introduced by GC are difficult to measure, but nevertheless an important component of performance.

pjmlp9• 11 years ago

Lets not forget they did that, because they failed to produce a stable working GC for Objective-C.

banachtarski13• 11 years ago

Reference counting is still not the same as manual memory management, and there is a heavy cost to reference counting as well. Reference counting trades throughput for latency relative to a garbage collected runtime.

pjmlp9• 11 years ago

Manual memory management only works in small teams with highly skilled developers, and even then there is a reason why so many CVE exploits are related to memory corruption.

Even those highly skilled developers aren't failure prof.

duaneb4• 11 years ago

Manual memory management (i.e. not GC) is a little disingenuous because it implies that it's entirely manual. In fact, there are many patterns (e.g. RAII) which make "brain dead" manual memory management possible.

I'm not necessarily arguing this is a great way to go for a new team, but there are manual memory management strategies that can definitely scale. Among other things, it's not the only type of resource that needs managing, and GC only handles memory.

banachtarski13• 11 years ago

Man, the new generation of programmers are going to be in bad shape if the thought of stack unwinding or managing the heap yourself is left to the "highly skilled." Even if I was interviewing someone who only worked in a memory-managed language, I would hope that they'd at least be capable of understanding how to code without it.

These should be the fundamentals.

decafbad• 11 years ago

presentation video: https://www.youtube.com/watch?v=PnPzeavMjmE

nickpsecurity• 11 years ago

Interesting article. What I see is another failure of industry to solve the root of the problem: different GC strategies work for different applications and should be selectable. Might even allow pluggable GC's. This is the path taken in JX Operating System and certain academic works on improving Java runtime. JX's first tier of memory management just gives a certain amount of resources a component/process/app can use. The second tier is its GC, which can be different per application. So, you could partition your application into components with associated GC's that focused on throughput, low-latency, or productivity. Then the problem shifts to making an optimal scheduler for the app and GC runtimes. There's already good schedulers, strategies for building them, and even tools for automated planning of scheduling strategies. Any of these might be integrated to optimize that aspect.

Overall, using pluggable garbage collectors, feeding a planner some constraints, and compiling the result sounds much easier that this person's day job. Embedded Java already does some of this while some systems did it in hardware to avoid most issues entirely. Enterprise Java should adopt such a method. Meanwhile, reading of these nightmares, I'll continue avoiding such GC-based tools wherever possible in my work.

JX Operating System for reference http://www4.cs.fau.de/Projects/JX/publications/jx-usenix.pdf

jokoon• 11 years ago

I wonder, isn't it working on the stack much faster than working on the heap anyway?

So when you query data on the heap, shouldn't you just query larger objects to put it on the stack, and work from there, instead of using the heap so often ?

With that in mind wouldn't that render garbage collecting almost irrelevant if your code is well designed, by not working too much on the heap ?

It's true that more ram makes the GC more relevant, but if it's an excuse for negligent software design, maybe GCs are not such a good idea. It's good to have features that make the job of the programmer easier, but if it only save the time of the skilled programmers who knows a little about how it works underneath, is it such a good idea ?

You can hardly convince that simple tools and predictable behaviors in machines are not the safest way of having fair results.

A programming language is already a big shortcut to work faster, and I doubt you should try to work even faster if it means creating new drawbacks.

outworlder• 11 years ago

I wonder, isn't it working on the stack much faster than working on the heap anyway?

Working on GC-managed memory is faster still.

Remember, not all allocations involve an actual heap allocation. Conversely, not all deletions cause a 'free()' operation on the heap.

vitalyd14• 11 years ago

Working on GC-managed memory is faster still.

Care to elaborate?

evanpw2• 11 years ago

The stack and the heap are both just regions of memory, so there's nothing inherently faster about using the stack. The difference is the memory management strategy: Usually, stack variables are all allocated at the beginning of a function, and all deallocated at the end. If all of the memory your program needs is tied to a particular scope like that, then it's certainly faster than any other memory-management strategy. If you need to share data between scopes, build complicated data structures of indeterminate compile-time size, or allocate large blocks of memory (the stack is usually limited in size), then you have to do something different (like use the heap).

That said, I think the Rust approach to memory management is really interesting: try to tie all allocated memory to some scope, and keep track of when ownership is borrowed by or moved to a different scope.

vitalyd14• 11 years ago

The inherent benefit to the stack is the memory region stays hot in cache due to natural use of the stack. That's your fast reusable buffer for temporaries.

In addition, of course, cleanup/reclaim of the stack space is pointer bump, so you get pointer bump allocation and deallocation, effectively.

pron17• 11 years ago

Sure, but on modern servers, the amount of data you can put on the stack (or stacks) amounts to no more a few percent of RAM. The smartest thing you can do with the rest, is put as much of your DB data there, and that data has an arbitrary lifetime, and is read -- and written -- concurrently. So most of the memory activity on large servers is going to be on the heap. If you put on the stack everything that can be done on the stack, you've reduced your heap usage by, say, 2%. True, the kind of memory work on the stack and on the heap is very different, but the two problems are of entirely different magnitude and nature.

I think the reason many people say "just use the stack more", is that most people delegate the hard work to a DB, and let the DB engineers worry about using the heap well. But the best way to use a DB is if it is in memory and in process.

dumael6• 11 years ago

Garbage collection worries about reachability of objects, not of access patterns. Some GC setups may not apply read/write barriers to stack objects as they are not required.

The difficultly with stack centric working is that you have to ensure that any object produced that is put back into the heap only references heap objects. Otherwise, by any sane language/style definition, you've made a mistake in referencing stack objects from the heap.

Accesses to the stack or heap, except in very specific circumstances[1] are equal but allocation/deallocation process are widely different.

Avoiding introducing GC work has/(can have) the tendency to "fight the language" problems. Consider writing Haskell code that does not allocate on the heap for example, it's damn near impossible. Otoh a Java program can shift some heap allocations into stack allocations.

[1] Afaik, some intel processors have small dedicated D-caches from stack relative load/stores. On the order of 128 bytes. I am probably wrong though.

jimrandomh• 11 years ago

Performance is nice, but correctness is more important. Complex programs written without any GC tend to have use-after-free and double-free vulnerabilities. Programs written using automatic reference counting only (without a way to catch cycles) tend to have memory leaks. If GC is killing performance, then that is a failure of the specific GC or programming language design. Don't blame the applications programmers for wanting GC semantics; that's a completely reasonable thing to want and something that language designers are capable of providing.

In the specific case of Java, I think the real problem is not having value objects (like C#'s struct) or generics with non-reference types. Serious GC problems happen not because GC is too hard, but because in Java an ArrayList<int> of a million items makes a million tiny objects. This problem is entirely Java-specific.

pjmlp9• 11 years ago

This problem is entirely Java-specific.

Fully correct.

Back when Java was still in its infancy, we had Oberon(-2), Component Pascal and Modula-3 as system programming languages that allowed for C like memory allocation, coupled with a GC.

But Java was the one that made GCs finally go mainstream and now many equate GC with Java ones[0], thinking all GC are made alike.

[0] Forgetting in the process that Hotspot is just one JVM among many.

astral3032• 11 years ago

Specifically about ArrayList<Integer> -- there are specifically a bunch of projects (Goldman Sachs Collections, trove, FastUtil, Koloboke) that solve the array of primitives problem.

I would say that's a Java-standard-libary-specific problem.

TheLoneWolfling5• 11 years ago

And then you have to do a copy just to pass it around to any other external libraries. And you lose the advantages of generics.

You end up with 9 copies of everything, that are 99% the same except for a couple find-replaces. If not more. (For instance, if you have a method that takes two generic arrays, you need 81 copies! Even if they are the same type you still need 36 (!) copies.)

Not a good solution.

kibwen• 11 years ago

  > that's a completely reasonable thing to want and 
  > something that language designers are capable of 
  > providing.

This is exactly what the designers of Rust have worked to provide: freedom from GC while maintaining guarantees against user-after-free and double-free, while allowing you to reference-count bits of data when you ask for it (RC leaks are rare in Rust--it's not trivial to create cycles--though still possible).

TheLoneWolfling5• 11 years ago

I don't see why people put so much work into GC before even bothering to do any work to avoid objects becoming garbage in the first place beyond simple escape analysis.

GC Tuning Confessions of a Performance Engineer

Related Stories