What exactly is Membase?

http://blog.northscale.com/northscale-blog/2010/07/what-exactly-is-membase.html

by mattyb • 16 years ago

15 3 16 years ago

blog.northscale.com

It looks like all reads and writes for a specific partition must go through the master node for that partition. Is that the case? If so, are there any plans to allow clients to read from the replicas if they are willing to get less fresh data? Another option would be to have Membase return a sequence ID as part of the result of a write to the master. Then a subsequent read request could present that sequence ID to a replica and have it block until either a timeout occurred or it was able to serve data at least as fresh as that sequence ID.

jamesmphillips• 16 years ago

currently reads/writes are to one master with load spread across the cluster. The hashing algorithm used for placement should statistically balance key activity across nodes. Thus, unless you have an extremely hot key(s) then you should see balanced utilization across available resources.

Having said that, there are cases where subsets of the keyspace get disproportionately "hot" and where load balancing reads would make sense. The vBucket mapping mechanism in membase was explicitly architected to support multi-read support in the future. But our focus initially was on providing a CA database that was immediately, versus eventually, consistent. We'll relax that constraint in the future. Did I say relax!? : )

ingenthr• 16 years ago

A couple of points. Some deployments do query from replicas already, but we can make this a bit better in the vbucket world. A second point is that moxi actually has functionality to non-coherently cache some items locally. Moving the cache closer to the application is always the best thing to do if you're willing to relax consistency. This is better than having it elsewhere in the cluster. The fastest network read is the one you don't actually have to do.

moxi does this with a key prefix and a timeout per item (even 1 second makes a big difference for hot data), then fetching it from the authoritative server.

We've not really exploited this moxi functionality in membase deployments yet, but it's been there in moxi for a year now and could be easily turned on if we had a deployment which needed it.

Having said all of that, with membase (as with memcached) the limit you'll hit first for an item you're fetching is network throughput. It's not that common to be fetching a few items so much that you overrun network. Even if you do, then the answer is simple: add a few more nodes.

What exactly is Membase?

Related Stories