The leadership myth in replicated databases (2023)

https://maheshba.bitbucket.io/blog/2023/05/06/Leadership.html

by shikhar • 2 years ago

60 3 2 years ago

maheshba.bitbucket.io

cljacoby• 2 years ago

I recently took a Distributed Systems course, and I also thought it was very interesting and unexpected how in the most basic form of Paxos, there is no concept of node roles or hierarchy like leader/follower, master/replica, etc. The base case is that all nodes have the same replicated log, and are "writers" capable of initiating changes to the log.

This youtube video was particularly helpful in learning about consensus algorithms, specifically Paxos/MultiPaxos:

https://www.youtube.com/watch?v=JEpsBg0AO6o

John Ousterhout (author of Raft) walks through Paxos/MultiPaxos as outlined by Leslie Lamport, and then talks about a series of optimizations to improve performance. One key optimization is transitioning from performing consensus on a single log-slot proposal, to the entire log-slot altogether, which mitigates failed consensus rounds, and is where concepts like leader nodes emerge. If your familiar with Raft and Paxos, while listening to these optimization applied to MultiPaxos, you can kind of notice it to begin to resemble Raft.

The course I took was through Georgia Tech, but was largely based around a framework developed at the University of Washington called dslabs:

https://github.com/emichael/dslabs

https://ellismichael.com/dslabs/452%20Discussion%20Section%2...

It was super informative for my learning about the foundations of distributed systems, namely consensus algorithms. I'd highly recommend it for anyone interested in learning more. Although fair warning, the programming assignments were quite difficult and time consuming.

kfrzcode• 2 years ago

Did you study anything about Hashgraph, or Hedera?

dboreham• 2 years ago

Thanks for posting -- hadn't come across dslabs. Nice to see it's written in a mainstream language (Java).

The leadership myth in replicated databases (2023)

Related Stories