Diskstore and a new b-tree implementation

http://antirez.com/post/diskstore-btree-implementation.html

by mattyb • 16 years ago

33 9 16 years ago

mumrah• 16 years ago

I could see one running out of inodes for a sizable dataset. Also, this seems like it would incur a good deal of iowait opening and closing all those fds.

antirez4• 16 years ago

Yes I guess the btree (or similar) will be much better for many small items, but the file solution may be the way to go for not su much items but much bigger...

Eventually maybe the fs solution can just be removed... it was a way to get started with the caching layer asap.

rb2k_• 16 years ago

Anybody tried using this on HFS+ in OSX? It was horribly slow for me, but it could also have been parameters in the conf that were bad.

antirez4• 16 years ago

By default the flush time is set to zero and the cache size to 0, so everything will be always loaded / processed / saved again, and ASAP. That said HFS+ is hardly cool...

agib• 16 years ago

Does anyone know the basic tradeoffs between the two stores?

snissn2• 16 years ago

Are you familiar with memcachedb? it's a berkleyDB back behind memcache with a lru key destruction in memcache that persists to disk using berkley db... you definitely don't need to re-invent the wheel here, you can possibly use innoDB as a key val store also if you can find the source/binaries for it....

antirez4• 16 years ago

In Redis the use of the disk KV store is just one of our layers, live objects are encoded in a different way to allow the usual Redis data model, so memcachedb is not a replacement for this unfortunately.

Both BDB and innoDB are mature implementations of on disk KV stores, but unfortunately there are licensing issues in one case, and code not clean / bloated in another case preventing me from using this solutions.

Basically my reasoning is always: if there is a good supported library that I like, I can use this, otherwise it is probably more interesting to learn something new. When I'll write recovery tools, deal with bugs, and so forth, it is better to deal with your own code if there isn't something of readily available, sanely licensed, simple to undertand, and small enough.

snissn2• 16 years ago

you should connect with the tokyo/kyoto cabinet author if you aren't already, http://twitter.com/#!/mikio1978/status/27344938710601728, his kyoto cabinet network 'daemon' is under heavy development and it would be really cool if you two were to collaborate and share code

Diskstore and a new b-tree implementation

Related Stories