Knossos: Redis and Linearizability

http://aphyr.com/posts/309-knossos-redis-and-linearizability

by m0nastic • 13 years ago

182 11 13 years ago

aphyr.com

mrbbk• 13 years ago

Incredible.

kellabyte• 13 years ago

+1 @mrbbk Incroyable.

wheaties• 13 years ago

I love Redis and have a ton of respect for him, his accomplishments, and what he's given the community. I now how a ton of respect for someone else. We all make tradeoffs. Redis is great at what it does. I will use it for that.

sillysaurus2• 13 years ago

That... is an impressive post.

mambodog• 13 years ago

Kyle really is a high-output kinda guy, isn't he?

abolibibelot• 13 years ago

That's a glimpse of an alternate universe where technology choices were backed by actual proof. Well done Kyle Kingsbury. Moreover, all his Jepsen series are a great writeup on distributed DB theory and issues, and nurture an healthy skepticism to the wild claims DB authors/vendors throw around.

http://aphyr.com/tags/Jepsen

sudhirj2• 13 years ago

Much respect for both Antirez and Aphyr.

I keep thinking Antirez is being forced into these problems by community pressure, though. There seems to be a lot of effort going into trying to make Redis some kind of super-featured distributed mothership of a database when we already know that it's not going to be possible.

Antirez and the community have put in an incredible amount of effort to create a datastore that is exceedingly good at what it does, probably best-in-class. To start asking for a distributed system with more guarantees than it was fundamentally designed for is unreasonable. The WAIT command, like every other command in Redis, was written to serve one purpose and do it well in bounded time. I expect that purpose was chosen keeping in mind what was possible without completely demolishing the current Redis architecture. If we can build a reliably distributed system using it, that's fine, but I don't think we ought to be demanding it this way.

antirez• 13 years ago

Hello sudhirj,

the model tested here was again nothing planned to be implemented, just a model to show that WAIT is just a low-level primitive, and the real behavior of the system is determined by all the parts of the system, not WAIT per se.

Redis Cluster is not going to be a strongly consistent system, I'm not going to be tricked into designing something is not good for the Redis Community in my opinion. However Redis Cluster tries with heuristics to put bounds on how much a replica can diverge, and to pick the history that makes more sense when a slave is promoted.

Btw, about the Aphyr post, 13 days ago I wrote in the Redis mailing list the following message in reply to him:

---

About non linearizability, perhaps it does not apply to the case where a strong coordinator exists, but in the general case one issue is that when we read, we can't just read because a stale master could reply with stale data, breaking linearizability. There is a trick to force the read to be acknowledged that could work:

    MULTI
    INCR somecounter
    GET data
    EXEC
    WAIT <n>

---

The fact you can't read from the master without care is a very obvious and well understood problem in distributed systems. Even a DS newbie like me was able to spot that in 5 minutes during the mailing list thread :-)

sudhirj2• 13 years ago

While I think the atmosphere could be a lot more co-operative and cordial, I think a lot of good work has been done anyway. The post may have attacked a straw man view that you've already discarded, but there's no denying that everyone's gotten a potentially very useful tool out of it :D

This is personal opinion, of course, but I think Redis would be better served by not striving to make formal guarantees of any sort. Instead it could continue to provide primitives while staying fast and stable. Heuristics should be acceptable and gradually improved as more approaches become feasible. This seems to be the approach being taken right now - I'm hoping you and everyone else can let that be.

hollerith• 13 years ago

Someone help me out: does the "DS" here mean data science?

Knossos: Redis and Linearizability

Related Stories