How to save the world with ZFS and 12 USB sticks: 4th anniversary video (2011)

https://constantin.glez.de/posts/2011-01-24-how-to-save-the-world-with-zfs-and-12-usb-sticks-4th-anniversary-video-re-release-edition/

by mariuz • 10 months ago

99 41 10 months ago

constantin.glez.de

oofbey• 10 months ago

It’s funny which things have changed and which haven’t. That server which was super impressive for 2007 had 4 cores and 16GB of ram. Which is a reasonable small laptop for today. The 24TB of disk would still be pretty big though.

bombcar2• 10 months ago

24 TB is available on a single drive now, though.

temp08262• 10 months ago

I haven't kept up with spinning rust drives so I had to take a look. Seagate has a couple 30 tb models now, crazy. Lot of eggs in one basket...and through the same ol 6gbit sata interface these must be a nightmare to format or otherwise deal with. Impressive nonetheless

CTDOCodebases3• 9 months ago

There are 36TB hard drives available.

There are 122TB SSD drives now, though.

hugmynutus• 10 months ago

Buddy, I have 24Tb HDDs in my pool today.

If anything the opposite has occurred. HDD scaling has largely flattened. Going from 1986 -> 2014, HDD size increased by 10x every 5.3 years [1]. If anything we should have 100Tb+ drives if scaling kept going. I say this not as a but there have been directly implications for ZFS.

All this data stuck behind an interface who's speed is (realistically after a file system & kernel involved) hard limited to 200MiB/s-300MiB/s. Recovery times sky rocket. As you simply cannot re-build parity/copy data. The whole reason stuff like draid [2] were created is so larger pools can recover in less than a day by doing sequential parity & hot-spairs loaded 1/N of each drives data ahead of time.

---

1. Not the most reliable source, but it is a friday afternoon https://old.reddit.com/r/DataHoarder/comments/spoek4/hdd_cap...

2. https://openzfs.github.io/openzfs-docs/Basic%20Concepts/dRAI... for concept, for motivations & implementation details see -> https://www.youtube.com/watch?v=xPU3rIHyCTs

godelski3• 9 months ago

Not quite that level, but you can get 8TB nvmes. You'll pay $500 a pop though...[0]. Weirdly that's the cheapest NewEgg lists for anything above 8TB and even SSDs are more expensive. It's a gen4 PCIe M.2 but a SATA SSD is more? It's better going the next bracket down but still surprising to me that the cheapest 4TB SSD is just $20 cheaper than the cheapest NVMe[1] (a little more and you're getting recognizable names too!)

It kinda sucks that things have flatlined a bit, but still cool that a lot of this has become way cheaper. I think the NVMes at these prices and sizes really makes caching a reasonable thing to do for consumer grade storage

[0] https://www.newegg.com/western-digital-8tb-black/p/N82E16820...

[1] https://www.newegg.com/p/pl?N=100011693%20600551612&Order=1

ghshephard• 9 months ago

Nowadays most cloud customers jam their racks full of absolutely vanilla (brand is mostly irrelevant) 128 vCores / 64 physical cores, 512 GByte servers for ~$18k - w/100 GBit NICs. That Sun 4500, maxed out (with 10 Gbit/Nics), sold for $70k. ($110K in 2025 dollars).

What's (still) super impressive was the 48 drives. Looking around -the common "Storage" nodes in rack these days seem to be 24x24TB CMR HDD + 2 7.68 TB NVME SDD (and a 960 GB Boot Disk) - I don't know if anyone really uses 48 drive systems commonly (outside the edge cases like Backblaze and friends)

LaurensBER2• 9 months ago

I just grabbed a deal on a 8GB / 2 vCore VPS for 15 euro per year. It's absolutely insane how cheap hardware has become.

At my first job we paid 6 figures for a 256GB ram machine. Now I see homelabers grabbing old servers with that much memory for their hobby projects.

klinch• 9 months ago

May I ask where you got such a cheap VPS?

davekeck• 9 months ago

Surprised that folks were still using RealPlayer in 2011

zenmac3• 9 months ago

Can someone here on hn with more in deepth knowelege about ZFS commenting on why it is superior to EXT4 for example for file storage? Does each dir handle more children for example?

Last time I read here HN ZFS still seem have edge case bugs. Has it matured now? Why don't distro such as debian etc just ship ZFS as the default instead of ext4?

ksk23• 9 months ago

I can only imagine the ZFS license is not free enough for Debian (not a rant).

em-bee• 9 months ago

the problem is less how free it is. as far as i know it is free enough. but the license is simply incompatible with the GPLv2 and therefore a combined program may not be distributed, because by distributing it you would violate the GPL license of the kernel.

davisr• 9 months ago

Quit being a lazy bum and use a search engine to answer your questions.

zenmac3• 9 months ago

I don't know, I find HN is one place were we can actually engage real humans conversation on these type of tech related questions. Human dialog is often something that can results in more interesting discovery. Don't think you can get that with agenatic RL question answers feed back loop between models.

necovek2• 9 months ago

ZFS is a combination of actual file system like Ext4, LVM and MD (software raid) subsystems on Linux, with extra features on top which you unlock when these are a single system.

Due to licensing, it can't be included in Linux kernel directly, and so it's not seeing the same level of scrutiny like the rest of the kernel, but it is arguably used in production more successfully than btrfs (comparable feature set, in-kernel, but not maintained as well anymore).

yjftsjthsd-h• 9 months ago

Can someone here on hn with more in deepth knowelege about ZFS commenting on why it is superior to EXT4 for example for file storage? Does each dir handle more children for example?

I'll tell you why I use it: Built-in compression, and data checksums (so it can catch and potentially correct corruption). Both are extra useful on storage arrays.

Last time I read here HN ZFS still seem have edge case bugs. Has it matured now?

The only significant bugs I've heard of in a long time are with encryption. Still not ideal, but not a show-stopper.

Why don't distro such as debian etc just ship ZFS as the default instead of ext4?

(The following is an oversimplification; I'm not a lawyer, so take with grain of salt.) There is, or at least may be, a licensing conflict between ZFS under the CDDL and Linux under the GPLv2. Both good open source licenses, but it is at best unclear that it's legal to distribute binaries that mix code under the two licenses. This makes it at best really messy to distribute a Linux distro using ZFS. (The generally accepted solution is to compile on-device, since the problem only happens with binaries.)

zenmac3• 9 months ago

Thank you for pointing out about the CDDL license. Encryption is not a show-stopper.

How to save the world with ZFS and 12 USB sticks: 4th anniversary video (2011)

Related Stories