Visualising the Go garbage collector

http://dave.cheney.net/2014/07/11/visualising-the-go-garbage-collector

by davecheney • 12 years ago

148 7 12 years ago

dave.cheney.net

zameericle• 12 years ago

great tool and and excellent code read. Thanks for sharing.

Intermernet• 12 years ago

Nice! and in 325 LOC :-)

This tool (or something like it) has been asked for a few times on Stack Overflow. I can see that many people will find this useful.

stickupkid• 12 years ago

I just opened up a PR[1] for live updating of the graphs (with a delay from the setTimeout from javascript side). It's not amazingly coded, but works for me atm.

[1] https://github.com/davecheney/gcvis/pull/3

userbinator• 12 years ago

The vaguely sawtooth-looking waves are pretty characteristic of all GC operation. If you look at typical GC graphs of the JVM or CLR, they look very similar - but this one has a difference I'll try to explain below:

In this case the workload doesn't look particularly suited to GC - notice the scvg.inuse line (light blue) spends more time at its lowest value than climbing or at the top, meaning that more time is spent holding onto memory that is not in use than actually using it... the difference between scvg.consumed and scvg.inuse essentially represents wasted memory. Estimating roughly, the process appears to be using 100% more memory than it needs to for 66% of the time, and 20% more for the other 33% for an average of ~70% (you can use an integral to get the exact average.)

Most of the JVM GC graphs I've seen are much "pointier" at the bottom, indicating that immediately after a GC cycle the allocations resume again so there is far less time where memory is allocated but not in use. The average between used and allocated memory is closer to 50% in that case.

jerf• 12 years ago

It looks like the sample benchmark is of the "godoc" tool in http server mode. In this mode, godoc serves a local website that is a near-clone of http://golang.org/ , which is particularly useful because the local "Packages" tab will contain not just the standard library, but all local packages, including the ones you've downloaded and the ones you are writing [1]. Consequently, the program is, on average, doing nothing. To get those graphs at all, I assume he was clicking around in the served web site, because otherwise I'd expect nothing at all to happen.

So we're not looking at a big website serving dozens of requests per second... we're looking at a program with a relatively expensive startup (and in this case I mean "relatively" literally; in absolute terms it's quite fast in my experience, I've never been able to start the program and ALT-TAB to my web browser and load a page before it's ready), which then twiddles its thumbs.

To compare it against an active Java server program, you'd basically collapse the whole graph into about 2 or 3 seconds worth of activity, if that.

[1]: Incidentally, another example of "something simple Go does that intuitively feels like it shouldn't be a special thing because it feels like every language ought to already have such a thing, except that when you go looking for it, you can't find it". It doesn't sound like an all-in-one API documentation of everything you've got locally would necessarily be that especially useful, but it really is.

4ad• 12 years ago

Note: an idle godoc does almost nothing, this is godoc started with the indexing option.

twotwotwo• 12 years ago

An interesting GC-related find earlier this week: it can help a lot to allocate in large chunks when you can:

https://plus.google.com/+ChandraSekarS0/posts/MNMWAXr63qz

Note that the entire chunk stays "alive" while any element in it is reachable, so this only works when arena allocation would in a non-GC'd language, i.e., when you're going to free the entire chunk at once (or in those rare cases you never need it reclaimed, e.g., 'cause it's a big fixed-size pool that you'll recycle forever).

To my surprise, allocating big chunks performed better than a Pool or manual recycling in this case. My best guess, and the author's, is that fewer allocations meant less space on memory-allocator bookkeeping (so fewer GCs) and less time spent sweeping in each GC.

If anyone's more aware of the guts of the runtime than I am and can confirm or disconfirm or elaborate, I'm curious.

Visualising the Go garbage collector

Related Stories