Benchmarking Integer Compression in Go

http://zhen.org/blog/benchmarking-integer-compression-in-go/

by zhenjl • 13 years ago

5 6 13 years ago

Please, please use bar charts. Line charts are suitable only for data series where the X axis is scalar, not separate categories.

Also, the article has absolutely nothing about testing methodology. I'm guessing you're using Go's built-in benchmarking, but it still needs to be clarified; are the timings the mean, 95th percentile, or what? What about variance -- better CPU cache usage would lead to less variance, I would guess?

Interesting article, though. Thanks for writing it.

zhenjl3• 13 years ago

Thanks for commenting!

And you are absolutely correct about the bar charts. Not exactly sure why I chose line but they have been replaced now.

The testing didn't use the built-in benchmark mechanisms but it did use the *testing.T mechanism. It's basically runs of different algorithms on the same set of pre-loaded integers. And then I recorded the time (ns) it took to run compression and decompression (time.Since(now).Nanoseconds() where now was time.Now()).

The caching effect should be pretty similar for all the algorithms since they work off the same LARGE slice of data, afaict.

I will update the article to reflect the testing methodology (may be a few days before I can get to it though.)

THanks

lobster_johnson3• 13 years ago

That's so much better.

PS. I noticed you have an article about bitmap indexing. I hope you realize that the algorithm you're using is based on WAH, which is patented by the DoE.

Benchmarking Integer Compression in Go

Related Stories