Experience with ePaxos: Systems Research using Go

http://da-data.blogspot.com/2013/10/experience-with-epaxos-systems-research.html

by ste5en • 13 years ago

80 19 13 years ago

da-data.blogspot.com

assface• 13 years ago

Paper weaknesses
Evaluated using prototypes of protocols in Go

Retarded. This is why I left academia.

dgacmu7• 13 years ago

Eh. (blog author here). It was exceptionally annoying, but in the end, the paper got in. You have to develop a thick skin, but I've found that in the long run, the process works pretty well. The feedback we got from our first submission to OSDI was awesome, and helped seriously improve the quality of the research, not just the paper. (It led us, for example, to adding the formal proof and the machine-checkable proof.) The NSDI feedback was crappy. I'm disappointed, because I like NSDI a lot. I think people were just having a bad-Paxos day. The final SOSP feedback was again really good.

fintler• 13 years ago

I'm someone who's happy you put the proofs in there.

As soon as I read "has no designated leader" I stopped reading and skimmed through to make sure safety and liveness proofs were in there somewhere before I spent any significant time reading the rest.

This feels like it's probably a nice advancement over S-Paxos and I'm looking forward to getting through it -- thanks for making it accessible.

Also, this is awesome: "Copyright is held by the Owner/Author(s)"

nano_o• 13 years ago

Great work. Do you have a link to the machine-checked proof? In the technical report there is only a formal specification and a detailed but not machine-checkable proof. It would be really great to see systems researchers write machine-checked proofs :)

tomp3• 13 years ago

I think it's a fair weakness. Since Go has a mark&sweep GC, which runs at unpredictable moments, for unpredictable durations, the performance and responsiveness of programs can be significantly affected by unpredictable factors, like small code changes, which might or might not trigger the GC. That is not to say that GC is inherently slower than C++ malloc/free or reference counting, but only that small changes in code can lead to large, unpredictable changes in performance.

steveklabnik• 13 years ago

The article even mentions this explicitly:

Getting the performance variation down was a little tricky. In several spots, we had to think carefully (and experiment with some dead-ends) about how to reduce the amount of garbage we were generating.

They do say it was worth it in the end, and that other languages would have similar drawbacks.

Daishiman• 13 years ago

How does Go compare to modern C++, using the newer feature and libraries without the old cruft?

dgacmu7• 13 years ago

From a systems perspective: Dealing with lots of communication and having lots of concurrency _where the concurrent actors are doing a lot of different stuff_ is easier in Go.

Achieving high performance on a single machine and building reusable data structure libraries is easier in C++. Things like vectorization, cache-optimized layouts, etc., are all more easy here (or just in C).

JulianMorrison• 13 years ago

What were the general techniques you learned for optimizing Go?

tomp3• 13 years ago

building reusable data structure libraries vectorization, cache-optimized layouts

Could you explain a bit more about this features of C++? I believe that you're referring to templates for the first point, but I'm unsure about the rest - SIMD? stack-allocation/memory-pool-allocation?

micro_cam• 13 years ago

This mirrors my own experience using go to write software used in biological research. Performance is good enough and the quickened dev cycle due to less verbosity and excellent tooling makes implementing variations on algorithms easy.

Ie I wrote a random forest implementation [1] and i've been able to quickly add in a bunch of different published and experimental variations on the algorithm as needed for the data sets I work on.

[1] https://github.com/ryanbressler/CloudForest

Experience with ePaxos: Systems Research using Go

Related Stories