Using LVM’s new cache feature

http://rwmj.wordpress.com/2014/05/22/using-lvms-new-cache-feature/

by 2bluesc • 12 years ago

57 7 12 years ago

rwmj.wordpress.com

I have to admit that for a moment I thought: awesome, there is a built-in cache feature now for the llvm compilers and it took me almost a paragraph to recognize that the author was not talking about compilers....

hf• 12 years ago

Excellent! I was just investigating btrfs[0] and realized that it doesn't offer anything like ZFS's "2nd level storage" (L2ARC) which is doing precisely this: keep what is nowadays referred to as hot data on dedicated (read: solid state) storage; as much as possible.

In the course of this little recherché I hit upon bcache and dm-cache but found (Debian) user-land to be entirely lacking (`modprobe dm-cache` doesn't raise an eyebrow, though).

Having dm-cache support in LVM -- one of those projects that figure centrally in my personal sysadmin lore -- would be just heavenly.

[0] Balanced Tree FS. Similar in outlook and approach to SUN-now-Oracle's ZFS and -- oh, irony of ironies -- also under the Oracle umbrela.

koverstreet• 12 years ago

Annoying thing about have a filesystem and a separate block layer cache (bcache or dm-cache) is they're duplicating a lot of the same work - filesystem maps from inode:offset to physical offset, cache does the same thing. Both doing their own allocation/reclamation/etc.

I've been turning bcache into a full blown filesystem in my spare time. It's working now - it's fast, and the code is _small_ compared to other filesystems.

Not production ready yet and it'll be awhile before I freeze the on disk format, though.

ChuckMcM• 12 years ago

Interesting stuff. I get a bit skittish around storage caches because of some scars from some pretty spectacular failures, but when they are working as designed they are quite awesome.

Multiple writers with some using ftrunc(2) that is the nightmare scenario for me. Would be interesting to do a walk through of considering each disk blowing up and the recovery from that.

edwintorok• 12 years ago

How well does this work with a database workload? Did anyone test PostgreSQL, or SQLite (in WAL mode) with this?

rwmj• 12 years ago

It's a bit surprising that this routine post investigating a new LVM feature turned up on HN and several other high profile sites last night while I was out at a gig.

So I'm afraid I haven't yet had time to do proper testing. The benchmarking in the post is next to useless. I will be looking at how virtual machines behave on top of LVM cache (versus on top of an uncached hard disk), but that will take some time.

voidlogic• 12 years ago

I would really like to see random IO benchmarks vs. bcache... Does anyone know of any?

Using LVM’s new cache feature

Related Stories