Recycling memory buffers in Go

http://blog.cloudflare.com/recycling-memory-buffers-in-go

by jgrahamc • 13 years ago

150 42 13 years ago

blog.cloudflare.com

thirsteh9• 13 years ago

This is nice, but I can't help but feel it's a little wrong that we're still making free-lists like in the olden days.

tptacek2• 13 years ago

You absolutely don't need to do any of this to write Go programs that outperform Ruby or Python.

thirsteh9• 13 years ago

Generally, sure. But it's still pretty easy to write garbage-generating Go applications that exhaust memory because the GC isn't compacting. In some applications this is a nobrainer: use a free list. In some others, like when you don't actually know what kind of buffer size you'll need, it's not so easy (although you could argue you should just fixed-sized chunks for whatever you're doing, and recycle those, but that's a lot of work.)

What I'm saying boils down to: Free lists are definitely useful, but the GC could be better.

zwieback2• 13 years ago

The olden days are not that old. I find myself using static arrays, free lists, malloc and its OO brethren as well as GC when available. All of those things have been around for as long as I can remember.

It's nice not to have to worry about freeing allocated memory but if you write a lot of low-level code there's a certain level of unease about letting the language or OS take on something that can have such a large performance impact.

jgrahamc2• 13 years ago

There's a proposal to add something like this in Go itself: https://code.google.com/p/go/issues/detail?id=4720

stock_toaster• 13 years ago

unfortunately, it looks like it isn't going anywhere right now:

https://groups.google.com/d/topic/golang-dev/QtSIOFsNFrQ/dis...

stcredzero5• 13 years ago

Buffer caching, the technique that never goes away.

GC does not mean your memory management is solved. It means that your memory management works well enough for development and sometimes for undemanding tasks. That said, generational GC can be like mediocre buffer caching "for free." But this still doesn't make all explicit buffer caching go away.

Given that buffer caching is "the techique that never goes away," why isn't some facility for handling this built into the runtimes of languages with GC? You could tell the GC that this particular type of object needs to be cached, with a cache of this size and growth policy, and then programmers never have to write and debug code that does this.

EDIT: Another way to think abut this: because GC is designed to be universal and foolproof, it's easy to get GC into a corner case where its behavior is very suboptimal. In the case of "memory pressure" you can think of this as the GC having to scan the graph of references way more often than it really needs to. However, the optimization techniques for many of these are fairly generic. What if optimization techniques could be built into languages and runtimes, then instrumentation could be used to empirically determine what optimizations are actually needed, then these could be simply "switched on" for particular classes or types.

fiatmoney• 13 years ago

That shouldn't usually require any explicit support from the runtime; it can exist as library code (and does - for instance Apache Commons: http://commons.apache.org/proper/commons-pool/ ).

stcredzero5• 13 years ago

That shouldn't usually require any explicit support from the runtime; it can exist as library code

Well, of course. People writing their own logically implies that such can exist as library code. But if you build in support from the runtime, then it might be possible to squeeze out a bit more performance. For example, perhaps it's possible to tell the GC that certain objects don't have to be considered as roots. (And to the extent that this is possible with current libraries, it's because there is support built into the runtime.)

EDIT: Also, having these mechanisms as library code probably opens up more opportunities for introducing implementation errors.

kenshiro_o• 13 years ago

Is the Go GC Collector not good enough so you have to resort to this? Would be good if you had the option (like in Java) to specify heap min and max sizes - and if set to the same value your heap would never resize.

zwieback2• 13 years ago

Maybe I didn't read the code right but where does it check whether a retrieved buffer is the right size? The makebuffer() routine appears to generate random size buffers but that length then doesn't get checked.

My experience from embedded systems is that buffer rings are great if you can predict two or three common sizes that cover your typical allocation needs.

I think even for people that work mostly in GC environments it's a good idea to read through the basic mechanism of heap management and allocation strategies. This article is a good intro to the basic idea and I like the charting and analysis being done.

jgrahamc2• 13 years ago

The sample code does not check the buffer size. This is a slightly artificial example to show the technique and have some interesting graphs to back it up.

felixge2• 13 years ago

FWIW: Node.js uses a buffer cache like this by default: https://github.com/joyent/node/blob/master/lib/buffer.js

thirsteh9• 13 years ago

It's a good thing Node.js doesn't have to worry about synchronization between threads... :)

felixge2• 13 years ago

Would the down-voters care to explain? I've pointed out how another platform deals with allocating raw memory buffers as it's relevant to the discussion.

I've made no claims of it being superior, and in fact, am largely using Go myself these days.

thirsteh9• 13 years ago

Using a shared buffer list isn't very efficient in a language with concurrency since you have to synchronize to take from/add to the list.

wzsddtc• 13 years ago

coincidentally …. this is how we pooled the buffers on Vine for Android to make Camera initializations efficiently.

thrownaway2424• 13 years ago

This technique puts a lock in your serving path which will necessarily limit parallelism. Go channel operations are not "non-blocking" in every sense. Even if you have them in a select block your thread has to take the mutex to find out if the buffer is empty (on receive) or full (on send).

Recycling memory buffers in Go

Related Stories