How to Build a Minimal ZFS NAS Without Synology, QNAP, TrueNAS (2024)

https://neil.computer/notes/how-to-setup-minimal-zfs-nas-without-truenas/

by 4diii • 16 days ago

346 233 16 days ago

Khaine• 15 days ago

Can anyone recommend a good server for a homelab to use for a storage purpose like this?

bombcar2• 15 days ago

It's a horrible idea likely, but I have an ancient old Dell PowerEdge R510. Probably sucks way too much energy, but it does what it does and the price of SSDs have skyrocketed so I'm not touching it.

burner420042• 15 days ago

I'm sure there's better options now but the HP ProLiant MicroServers (used).

They support ECC ram, 4 caddies, one extra PCIe slot, and to my knowledge you're not cpu limited for a zfs file server usecase.

Keep in mind though, all you need is linux* support, iDRAC, ECC if you're a snob, and drive bays ... and that's basically any free server.

In my extremely opinionated opinion I would only get used enterprise server gear, because a zfs file server will just work unless hardware fails. And a UPS.

*ZFS will be a more natural choice on FreeBSD. It's better documented, and will meet Linux 1:1 in hardware compatibility for this.

bpye2• 15 days ago

I'm still running an old Gen 8 MicroServer. Modern drives can actually saturate the SATA controller, and because it only has a single PCIe slot I can't add both a 10Gb NIC and a storage controller - I went with the 10Gb NIC.

It works well enough though and has lasted me over a decade at this point. 16GB DDR3 ECC, an old 4 core/8 thread Xeon, 4x14TB drives and the Mellanox NIC.

ahepp2• 15 days ago

I got a gen10 plus microserver and liked it so much that I got a second one.

Throw FreeBSD on it and add a couple lines to /etc/exports and rc.conf and it's a NAS right out of the box

itchingsphynx• 15 days ago

Agreed. I have used a HP N40L ProLiant MicroServer since 2013 as home NAS and Time Machine backups via samba. Rock solid hardware, incredibly expandable, and today runs FreeBSD 15.1 with ZFS. Additional hardware modifications include; CD-ROM replaced by two 3.5" HDDs on mounts (now six HDDs of 10TB each in ZRAID1), a SAS card to add two mirrored bootable SSDs underneath CD-ROM drive space, a 2x 2.5G NIC (limited to 4GB/s slot) for dedicated NFS link, while additional internal SATA and external SATA ports unused. Next: replace PSU fan with quieter Noctua fan.

simondotau• 15 days ago

Not strictly a recommendation, but Terramaster is a good brand to look at if you want Synology-shaped hardware which can run TrueNAS or Proxmox or any flavour of Linux you want.

Along with various other devices (including a large Synology which I wouldn’t buy today), I run Proxmox on a small two bay+two nvme Terramaster. I have a bare bones Ubuntu LXC running Samba configured for Apple Time Machine, an VM running Scrypyed, and PBS for Proxmox backups. Nothing on it is critical so I don’t bother with any storage redundancy.

Fr0styMatt883• 15 days ago

I have the F8 Plus, great little unit. It did need a BIOS update when I first got it to enable Proxmox/other OSes to work properly.

Recently replaced the internal USB boot drive with a small NVMe USB enclosure; using a 90-degree USB connector and using a dremel to sand away an opening for the cable to come out so I could mount the enclosure externally.

KaiserPro2• 15 days ago

Depends on what you are after.

if you are after quietness and "power" then an old workstation is a great bet. They normally have space for at least 4 fullsized HDDs, the more modern ones have lots of PCI space for nvme-ssd cards (for space) and some have lots of lanes for speed

They also tend to come with SAS/whatever remote nvme is called/SATA expnasion options

The down side is they are not as space efficient, they also tend to have 60-120watt CPUs, so expensive to run

beagle34• 15 days ago

I still pay for snooty, and the reason for that is that when a disk goes bad (not if; when) I pop its tray out, replace the disk, pop the tray with the disk back in, click a couple of widgets, and that’s it. I know it will be rebuilt properly.

(And I know I have to do that, because when the disk fails it beeps and lights a led near the bad disk)

It’s easy to build a NAS such as the one described in this article, but in the long run, data loss is significantly more likely.

Also, any guide like this that doesn’t guide you through “disk 3 failed, this is how you safely replace it” is imho incomplete, even if it doesn’t go through telling you how you know a disk has failed.

msh• 15 days ago

Is snooty a autocorrect for Synology or some other product?

beagle34• 15 days ago

Indeed. Didn’t notice and it’s too late to edit now.

denkmoon2• 15 days ago

That is kind of exactly how zfs works though. The guide isn't complete, sure, but "rebuilding" the array is just replace the disk and run a single zfs command.

beagle34• 15 days ago

That’s comforting to know; that wasn’t true in the past for ext4 over lvm (is it true now?).

But what is that command? And how do you know which disk has gone bad?

I am sure I can get an answer from Google / Claude / ChatGPT, but a guide is incomplete without it - and the failure report should be active like a beep or flashing hardware light - I typically log into my NAS only a few times a year. A motd or other banner isn’t sufficient.

wyager• 15 days ago

ZFS makes this completely trivial except for the "beeps and lights a led" part

ahofmann• 15 days ago

To be fair, the "beeps and lights a led" part is very important in case of a dead disk in an array. The only time, I disrupted a service in production for longer than a few hours was when I didn't get the email of a degraded raid 1 array and the second disk died a few weeks later.

shric2• 15 days ago

smartmontools makes this trivial and can email you which might be better than beeping and lights if you’re not always near your server.

I use that plus smartctl_exporter so I can do metrics and alerting off that.

traches• 15 days ago

ZFS handles drive failures more robustly than anything else. There's a reason synology uses mdadm under BTRFS instead of the built-in BTRFS RAID features, and mdadm operates at the device level. That means that to replace a drive, mdadm has to rebuild the entire drive while zfs will only rebuild what's actually in use.

Also, any guide like this that doesn’t guide you through “disk 3 failed, this is how you safely replace it” is imho incomplete, even if it doesn’t go through telling you how you know a disk has failed.

`zpool replace my_pool disk3 newdisk`

bobmcnamara4• 15 days ago

The worst part of either approach is: how do I find disk number 39 out of 48.

splitbrain• 15 days ago

I came to the same conclusion when I built my NAS. Just Nix for the system, zfs for raid and docker compose for any service I might want to run. https://www.splitbrain.org/blog/2025-08/03-diy_nas_on_nixos

colordrops• 15 days ago

Same, using NixOS as a NAS, though it kept growing and now I'm trying to share it with other people. I use BTRFS on MDADM though. It's ended up being an all-in-one home server and router now because I was tired of the noisy power hungry rack I had in my closet.

https://HomeFree.host

fceccon• 15 days ago

I'm also using NixOS for my NAS, you should look into quadlets (docker containers managed by systemd) to manage containers with nix.

6ak74rfy• 15 days ago

But why? Why not the standard oci-containers module?

I've seen the quadlet-nix project but don't get what additional it gets me.

taude• 15 days ago

This is what I'm hoping to do. I haven't done much research yet, so I look forward to reading about your learnings. (Also to the person who responded to you with their experience and link.)

Thanks for saving me probably some hours...

EDIT: I'm also coming from an old Synology NAS from the ~2015 era.

Confiks5• 15 days ago

Not a great time price-wise to be building a NAS, but I have been doing so for the last two weeks. Inside a Jonsbo N6 case, which is pretty nice with an 8x SATA backplane and drive bays (unlike the earlier Jonsbo variants).

I ended up on shucking 4x the 14 TB WD Elements Desktop. They contain helium drives, the WD140EDGZ in my case, and are about a third cheaper than 4x the 12 TB WD Red Plus drives (which are air-filled). The shucking was easier than I expected too, and the performance seems very comparable. The warranty is a definite downside (European, so no Magnuson-Moss), but I think I can even get them back in their enclosure should they fail during the 2-year warranty period.

I've put some second hand 256 GB M.2 SSDs in there as boot drives. It was a bit of a struggle to get it to work in a way that failure of one of the drives doesn't hold up booting, combined with LUKS, TPM keys and ZFS on root. Learned a lot about systemd-boot which I have never used before, but feels a lot saner to me than grub ever was. So now I have a large script which debootstraps a Debian based NAS into being.

I noticed that there are a lot of ZFS myths and cargo culting. For example TFA mentions ECC RAM, which in some circles is a must-have because ZFS would wreck your pool during a scrub otherwise, which is a myth. It's also very expensive, especially this year. You also don't need much RAM for ZFS, L2ARC doesn't use much RAM at all, to name a few others.

Still doubting about setting `dnodesize=auto` (which is the default), because there are some horror stories about that [1]. And it seems impossible to find a cloud storage provider with reasonable prices that supports `zfs send`. Rsync.net upped their minimum order to 10 TiB recently, which is far too much for my use case.

[1] https://github.com/openzfs/zfs/issues/11353

[2] https://www.rsync.net/products/zfsintro.html

watermelon0• 15 days ago

I've been thinking about building my own NAS as well. Mind sharing how much did you pay for those hard drives, and what motherboard did you choose?

Confiks5• 15 days ago

It was €329 per hard drive (through a reputable store), and I chose the ASRock B650M-HDV/M.2 mATX motherboard combined with a Ryzen 5 8500G. Stock CPU cooler, replaced the Jonsbo case fans with Arctic P12 Pro PST LN. I only slightly regret the PSU (MSI MAG A650GL), which could be quieter. Not that it's very noisy, and it's a great PSU otherwise, but I should've chosen one that just shuts the fan down at low power usage.

conradev• 15 days ago

https://diskprices.com/ is great for this

You have SATA or SAS to pick from. The CPU requirements for a storage server are not high. On a typical ATX board you have motherboard SATA and can put SAS controllers in the spare PCIe slots.

My first "NAS" was two 22TB hard drives in a ZFS pool on my motherboard SATA

realityfactchex• 15 days ago

Not a great time price-wise to be building a NAS

That under-states the matter. It is a terrible time, price-wise, to build a NAS.

I'd almost rather have no AI whatsoever and have storage 1/10 the price of pre-AI times.

(If there were a magical choice between having AI and significantly more expensive storage, and having no AI and some program to dump that investment money into getting and somehow leveraging significantly more available storage, that is.)

bigstrat20032• 15 days ago

I would kill (figuratively, anyway) to have no AI whatsoever. No slop machine threatening to replace my job, or turn my job into babysitting its stupidity, and hardware would be reasonably priced? That would be awesome. AI has brought me nothing but downside.

randusername2• 15 days ago

I'm doing it anyway and I've found some secondhand deals. Maybe COVID homelabbers are offloading gear?

I built a 24 TB HDD NAS in a 36 bay chassis for around $1000 all-in: mb, ram, rails, chassis, rack, disks, hba, nic

Awful Watts/TB but the plan is to run it and the GPU rig on solar.

wolrah• 15 days ago

I'd almost rather have no AI whatsoever and have storage 1/10 the price of pre-AI times.

Almost? ALMOST!?!?!

If you handed me a button that would make it like LLMs had never existed I'd be slamming that button so hard Sam Altman's clothes would spin around.

Return memory and storage prices to normal, undo the sloppification of ~everything, remove all these annoying "features" that are so useful they have to force them upon people, and make scammers actually have to put in a slight bit of effort, all at the "cost" of real human developers, artists, writers, etc. getting paid for their work....

If this is a hard decision for you, you are the problem.

philjohn• 15 days ago

You're telling me.

I had a drive go bad in my 4 year old home-built NAS a few weeks ago, and it cost 2x what it originally did to buy the same capacity drive, and that was going to a grey-market importer on eBay, it was more on "reputable" sites.

nirav72• 15 days ago

Looking at current hd prices, I keep kicking myself for giving away several 3tb and couple of 4tb drives I had in my unraid server few years ago when I started upgrading my array to 8 and 12tb drives. At the time, a drive slot in my server was more valuable to me than a mere 3tb drive. i could have kept them for emergencies if any one of my array drives fail.

Hamuko• 15 days ago

I'm also planning a new home server build now and the prices are definitely relatively ass. So far I've spent 820€ on two 22 TB WD Elements HDDs, 375€ on 2x16 GB DDR5 kit, and 520€ on two 2 TB M.2 SSDs (cache). About 1700€ and I still have no server to show for it. Doesn't help that I've been procrastinating on picking the CPU and motherboard.

Fr0styMatt883• 15 days ago

I’m running an 8-drive ZFS RAIDZ2 pool. I’m wondering if you know — are the free space recommendations around ZFS cargo or real?

Like I’m already giving up two full drives for redundancy (which saved my ass - I recently had two drives fail on me in quick succession — both SSDs from what looks like an identical batch) but then the advice is kinda saying I need to keep at least another drive worth of space free for the pool to perform well and not crap itself. That hurts with current prices for sure.

gpt53• 15 days ago

ZFS will auto-degrade performance if there isn't enough headroom. In addition, if you use SSD, you also want the headroom, because otherwise you end up writing and rewriting on the same small empty space which kills SSD (unless you have enterprise SSD, which have built-in headroom).

Confiks5• 15 days ago

I’m running an 8-drive ZFS RAIDZ2 pool. I’m wondering if you know — are the free space recommendations around ZFS cargo or real?

I'm not entirely sure, but it seems to me that free space (and the 20% reservation) is mostly a proxy for fragmentation, and you can therefore better look at fragmentation directly. That would mean that if you mostly store large files, there shouldn't be a lot of fragmentation even at high utilization. The whole "ZFS changes allocation algorithm from 80% usage on" is something of 10+ years distant past, and lots of things around the allocator have been improved. It's also something that probably isn't too different from the performance of other filesystems at high utilization, so it shouldn't be exaggerated.

vladvasiliu• 15 days ago

I don't think they are. I've already filled a ZFS pool almost to the brim with no adverse effects.

cm21873• 15 days ago

For home usage, if you have backups raidz1 is fine (just do an incremental backup at the first sign of trouble). If you don’t have backups, then you probably shouldn’t be running a NAS in the first place.

neilpanchal3• 15 days ago

4x the 14 TB WD Elements Desktop.

I recommend staying away from SATA drives (huge consumer rush) and look for SAS drives on eBay, particularly HC520 Helium drives from HGST/WD. Need a SAS3008 PCIe adapter ($20) and a SFF-8643 splitter ($30). No backplane is required. Huge lots of decommissioned drives frequently appear. I bought Qty 10* of HC520 (12TB) SAS drives for $1000 about 3 weeks ago, avg age is about 2.5 years, still well within its rated lifetime.

Yea may be some of the stuff is fear mongering and cargo culting. I was told ECC is necessary for ZFS. When the article was written, it was cheap af (2024) to buy ECC RAM so not much consideration was given to it.

-- (*) I have no idea what to do with it. Anyone has any good idea for using 120TB space? I have about 40TB unused bandwidth in the datacenter, may be host a Debian mirror? Donate storage/bandwidth to Internet Archive? Please contact me, appreciate it.

Confiks5• 15 days ago

I bought Qty 10* of HC520 (12TB) SAS drives for $1000 about 3 weeks ago, avg age is about 2.5 years, still well within its rated lifetime.

That's cheap indeed. Enough headroom for some failing disks too. How is the noise and power usage? I didn't look at SAS drives at all, because my impression was that they're very noisy. I can place my NAS in a closed off room, but it's not too far away and I was afraid SAS drives would be audible through the wall. At the same time, the shucked drive I'm using presents itself as an WD Ultrastar, which comes very close to a SAS drive, and isn't very noisy.

ylyn• 15 days ago

You assert that ECC RAM being necessary for ZFS is just a myth but provide no justification for why that is untrue.

Is it not the case that if you don't have ECC memory, ZFS could end up writing a checksum that does not match the data if you get a bitflip in just the right (wrong) spot?

naturalmovement• 15 days ago

You assert that ECC RAM being necessary for ZFS is just a myth but provide no justification for why that is untrue.

ZFS without ECC is no more risky than any other file system / software RAID without ECC.

As no one owes you an explanation, it would take you five seconds to Google this and discover:

1. It's been disproven, with one of the original ZFS developers chiming in.

2. The original source of the rumor was a forum post that somehow became canon.

Confiks5• 15 days ago

Yes, indeed. ECC RAM is better than non-ECC RAM, also for ZFS.

The myth, popularized by a notorious thread on the TrueNAS forums [1], is specifically that ZFS requires ECC RAM, and will do worse than other filesystems without it, because scrubbing will multiply a single bitflip into a failed pool.

A ZFS core developer says that that isn't the case [2]. Here's some more reasoning [3], also about many other myths.

[1] https://www.truenas.com/community/threads/ecc-vs-non-ecc-ram...

[2] https://news.ycombinator.com/item?id=18480016

[3] https://kldload.com/zfs-wiki/myths

vardalab2• 15 days ago

Had it happen to my PBS backups running on ZFS without ECC just the other day. Turned out to be heat related, memory was fine but during long backups heat was causing bit flips.

nuker2• 15 days ago

combined with LUKS, TPM keys

Does it work? Server can reboot and use TPM to unlock rootfs? What about /boot - encrypted and tamper proof? Resists evil maid attack?

Rabbit hole, I know, but so fascinating if you solved it all :)

Gigachad• 15 days ago

If you are worried about someone breaking in to your house and replacing the bootloader while leaving your drives in place I probably wouldn't use the TPM auto unlock even if in theory secure boot should be able to handle this.

But in reality that will never happen and the only actual attack you need to be worried about is junkies breaking in and flogging the drives on facebook marketplace. For which, this level of security is fine.

toast03• 15 days ago

I run my cheap hosting box with a cleartext boot setup that uses ssh to automatically grab the key for the real root from my home server (or an alternate at my MILs house). Using FreeBSD, but similar concepts.

A previous hoster once gave me someone else's drives without wiping them. I don't want random customers snooping around on my data if a similar mistake happens with my disks.

For home use, I run without disk encryption. If I ever need to do data recovery, it's not going to be possible with encrypted disks and one point of a centralized NAS is to have stable long term storage.

NekkoDroid• 15 days ago

What about /boot - encrypted and tamper proof? Resists evil maid attack?
Rabbit hole, I know, but so fascinating if you solved it all :)

Encrypting `/boot/` is the wrong thing to do. One can just replace it with something that logs the encryption key instead (and sends it off to somewhere). You actually need to verify that what you expect to be there is actually there and a encryption isn't gonna do that for you.

A reasonably secure boot chain looks roughly as follows: bootloader is checked by the UEFI (read secure-boot), bootloader chain loads a signed UKI which also uses secure boot verification (or just skip the bootloader if you only have a single kernel with no fallback), this then loads a signed dm-verity `/usr/` which brings you to the login screen. Root is just data (e.g. mounted `noexec`) and is encrypted using e.g. PCR 11 so that it only unlocks if the UKI is the one that is expected (with some other backup unlock method).

This is basically #2 of the "Design Goals" of Poetterings "Fitting everything together"[0]. It's a good read if you are looking to designing your own Linux image and are looking for some inspiration. There is ParticleOS[1] which somewhat dogfoods this kind of system, though I don't think it is anything remotely considered for a production system.

[0] https://0pointer.net/blog/fitting-everything-together.html

[1] https://github.com/systemd/particleos

jsr0• 15 days ago

Have you looked at https://zfs.rent/ for cloud storage? You have to provide your own drive, but the monthly cost is relatively low.

devy• 15 days ago

Yep. The 4 Samsung SSD 990 PRO 4TB NVMe SSDs are $1100 apiece. Yikes!

https://www.samsung.com/us/memory-storage/nvme-ssd/990-pro-p...

tomasff• 15 days ago

So now I have a large script which debootstraps a Debian based NAS into being.

Is part of this publicly available? After seeing, - https://words.filippo.io/frood/ - https://0pointer.net/blog/fitting-everything-together.html

I've been looking at setting something up with mkosi/systemd for my home server/NAS - it's interesting seeing how different setups are solving the problem for inspiration.

tamimio2• 15 days ago

Yeah no. When it comes to backups and data storage, I would rather use a proven reliable system that’s been used and tested by millions of other users, keep these hacky stuff for your hyperland set up.

bombcar2• 15 days ago

I've lost more data to "proven reliable systems" than to my homegrown hacked-up stuff (which may be because I trust the latter much less).

thenthenthen• 15 days ago

This. My synology nas got pwned so many times i cant remember.

bakugo• 15 days ago

I would rather use a proven reliable system that’s been used and tested by millions of other users

You're describing ZFS.

tamimio2• 15 days ago

Not really, zfs is part of it, but not the whole picture, you as a user you need more than just a file system.

Truenas for example add replication, external cloud sync, gui (yes it’s important not to wipe your data with wrong command), HA, custom caching, containers to extend ot with other apps, raid-z expansion with openzfs, and other for monitoring, user management and a lot more.

Other solutions also add more than just a zfs with ssh access.

NexRebular• 15 days ago

You're describing ZFS.

..running on illumos.

aligutierrez• 15 days ago

this is really cool. I've been dealing with an aging Synology nas and this is something I can pick up, evaluate performance and how safe it is to serve as home for my data.

AceJohnny2• 15 days ago

Tangential, but about this:

I am creating a RAIDZ1 (RAID 5) zpool. That means 1 drive redundancy in-case of failure

A friend once told me that RAID5 has a high latency cost, because every Write requires a Read to update the stripes across all drives, and while this made sense when drives were expensive, nowadays you might as well do a RAID10 instead, and trade space for latency.

Is this still true with ZFS RAIDZ1?

bpye2• 15 days ago

and while this made sense when drives were expensive

I don't have the answer to the latency question, but HDDs have shot up in $/TB over the last couple years too. They are once again kind of expensive.

shrubble• 15 days ago

The RAID5 write hole is not present in RAIDZ1 I believe.

_nhh• 15 days ago

RAIDZ1 will add more stress to the consumer ssd's as to enterprise one due to surprinsingly different hardware. Consumer ssd's life times are calculated on a 8 hour shift (the tbw) and enterprise are calculated on 24/7.

Please - dont - use - consumer ssd's - with zfs raidz1

unsnap_biceps• 15 days ago

It's complicated, but https://jro.io/capacity/ actually walks you though how it works, towards the middle of the page.

``` Unlike traditional RAID5 and RAID6 implementations, ZFS supports partial-stripe writes. This has a number of important advantages but also presents some implications for space calculation that we'll need to consider. Supporting partial stripe writes means that in our 7wZ2 vdev example, we can support a write of 12 total sectors even though 12 is not an even multiple of our stripe width (7). 12 is evenly divisible by +1 (3 in this case), so we don't even need any padding. We would have a single full stripe of 7 sectors (2 parity sectors plus 5 data sectors) followed by a partial stripe with 2 parity sectors and 3 data sectors. This will be important because even though we can support partial stripe writes, every stripe (including those partial stripes) need a full set of p parity sectors. ```

rexysmexy• 15 days ago

To this day I still use a ZFS array for my critical backup storage. I've migrated over to using snap raid + mergerfs for my larger Linux iso storage array. Simple enough and I can pull a drive on it's own without any other stuff.

bradley13• 15 days ago

Just yesterday I was brainstorming with ChatGPT about this. I have an ancient QNAP plus a slightly less ancient NUC running PiHole, Wireguard and other services. Both need replaced, so why not combine them?

I don't know much about ZFS, but it sounds like I need to learn. Docker may have conquered the world, but I plan to stay with LXD for services.

The one thing I take issue with: an appliance like this runs 24/7. It should be low power and fanless. A processor like the N100 seems like the obvious choice.

bartvk• 15 days ago

It also surprised me that the author said "4 Cores, Xeon Server CPU can be had for cheap".

But the specs also said ECC RAM and I don't think the N100 supports that.

sunshine-o• 15 days ago

But the specs also said ECC RAM and I don't think the N100 supports that.

If I remember correctly it can in theory but in practice I have never seen a N100 with ECC.

layer83• 15 days ago

The N100 supports "In-band ECC" (IBECC), which uses regular non-ECC RAM at the cost of less available memory and a 10-20% performance drop. It’s unclear how well it works, and almost nobody uses it.

mianos• 15 days ago

I run zfs as the storage pool for my incus (next lxd) services. It is the ideal fit. Here is a list:

- Instant, zero-copy container cloning from images via Copy-on-Write. If you boot a new image like the existing ones it's seconds.

- Atomic, millisecond-level instance snapshots regardless of storage size

- Block-level container migration using native 'zfs send' and 'zfs receive', very short command lines and seems to work perfectly.

- Granular dataset nesting (every instance, image, and custom volume gets its own ZFS dataset). You can see every filesystem even on the host.

- Transparent, inline data compression (LZ4/ZSTD) enabled automatically per dataset. For services that don't change much, you might as well use a compressed image to make them even smaller.

- Mirroring / Raid

- Sub-volume sharing and direct management via native ZFS administration tools. If my home directory has a build area and a million files, I can just save time and put my home, pre cooked into a new machine and not copy or even rebuild on my new machine.

- Dedup keeps blocks with the same data as a reference. This costs a lot of memory and has not saved much for me as a lot of my images are similar and already shared I think, but it's cool.

throwaway2709252• 15 days ago

- Dedup keeps blocks with the same data as a reference. This costs a lot of memory

To get around this you can add an SSD mirror as "Special" vdev, the dedup table will then be saved there, freeing up your memory.

RossBencina• 15 days ago

I've been thinking about setting up something like this for a while. I have a Broadwell dual socket Xeon workstation that I'm going to upgrade to Proxmox. Would it be reasonable to run something like this as an LXC or VM or would you put it in the base kernel?

TacticalCoder2• 15 days ago

I have a Broadwell dual socket Xeon workstation that I'm going to upgrade to Proxmox. Would it be reasonable to run something like this as an LXC or VM or would you put it in the base kernel?

I've got a small HP Z440 workstation (small in that it's 10 years old and only single-CPU) and a 14 cores / 28 threads Xeon e5-2680 v4 in it. It's a little workhorse.

I run Proxmox (basically "Debian + ZFS out of the box + easy to spawn VMs / LXCs") on it.

And you can totally put something like FreeNAS in a VM. If you plan to really serve lots of files and need lots of perfs, you should get a tiny bit fancy and passthrough to the FreeNAS VM an entire disk controller (device passthrough to a VM in this day and age of LLMs that are there to find the relevant parts of docs for you in a split second is easy). IIRC the FreeNAS documentation itself explains how to run FreeNAS in a VM so I don't think it's "unsupported".

There's nothing wrong with running it bare either.

crowd51• 15 days ago

Very cool

jmusall• 15 days ago

This post mentions https://github.com/wyager/zfs-backup for (offsite) backups. Why would they recommend this over something more established like https://github.com/jimsalterjrs/sanoid? Has anybody tried it?

neilpanchal3• 15 days ago

It's pretty minimal and straightforward written in Haskell. Probably can write one from scratch with Claude now.

igoose12• 15 days ago

What's next!? Let Claude analyse clinic visits!? Oh well...

INTPenis• 15 days ago

One alternative for those who don't want any of the major NAS vendors, just use RHEL10. It's free up to 16 licenses, it's ultra stable, cockpit is a very mature gui for a lot of maintenance tasks.

It's a set and forget OS that will run for years without requiring your attention. But these days it has decent container support for hosting services on.

theNotFractured• 15 days ago

Or rocky, which is a 1:1 clone of RHEL that doesn't require licenses. https://rockylinux.org/

bigstrat20032• 15 days ago

The problem with RHEL is that the free version exists at the sufferance of IBM, a company not well known for being motivated by keeping tech enthusiasts happy. I would use Debian, personally. Not as long of a support cycle as RHEL, but still quite stable and no possibility of corporate rug-pulls.

kartoshechka• 15 days ago

I found cockpit to be incredibly useful for managing NAS and VPS. Even though any configuration is not an issue with LLMs, having a dashboard with toggles, formatted logs and such is helpful, and I can even run shell commands on phone without ssh. In the NAS case, you can easily see the results of backup/maintenance cronjobs, manage samba (which is a huge pain in the ass to get to work properly on iPhone), and monitor disk usage

embedding-shape• 15 days ago

and I can even run shell commands on phone without ssh.

How do you do the authentication to cockpit, basic HTTP auth in front or something? I've always ended up making my phone SSH-capable rather than dropping down to anything else but public/private keys for authentication to my servers, guess I have a bit of conspiracism when it comes to that.

Havoc2• 15 days ago

Also you can use zfs on a proxmox host and use that both for NAS duty and for the VMs.

For some reason people insist on doing truenas on top of proxmox and then introducing a networking layer between everything they do. Noooo…

lucasrufkahr• 15 days ago

its probably dumb but what i do is use proxmox with a debian VM with docker installed and each docker container gets a virtual disk i make on the proxmox host. obviously more involved than running unraid or truenas but i think the complexity underneath is simpler.

Havoc2• 15 days ago

Thats perfectly reasonable. Double isolation but that isn’t too costly.

I’m talking more about people doing the same as you except they’re linking the VM storage to another vm (truenas) over network despite it all being on same host. Think it’s mostly because people don’t want to deal with zfs via command line

giov4• 15 days ago

I killed 2x 4tb Nvme drives on Pciexpress with ZFS raid1 under proxmox, got them replaced, tried again on both pci-express and usb adapter kept getting io problems there too with zfs, drive gets disconnected during scrub or high load, tried already kernel or power management options for pciexpress and usb without sucess, managed to reproduce with high load.

or proxmox ( especially community edition) is fucked or zfs is still unstable, surprisingly with different drives on different interfaces I obtained same results, didnt swap proxmox for clean os, that might being some changes since modules and options would be different i guess. ( drives firmware etc are fine and performing well when inspected with proprietary windows tools)

just search zfs nvme pcie or usb problema or disconnect and you see similar stability problem for different users in different cases (os, drives etc), but also unraid and others, maybe somewhere is rithe right combo of options/glitches but didnt find it yet

65813• 15 days ago

usb adapter kept getting io problems there too with zfs, drive gets disconnected during scrub or high load

That's usually caused by the adapter chipset overheating. Don't use USB adapters for important data.

anon-3988• 15 days ago

I really, do not want to have a big server rack lying around at my house. I have a couple of SSDs that are not being used and 3 USB HDD drives. I also have a PN50 that I am not using as well.

I am thinking of buying a USB bay with 5 SSD slots in it and then 3 HDD drive. My use case is very, very cold. It is mostly just readonly data with some rclone sync every week.

Does anyone have a suggestion? I am pretty much relying on ZFS to do all the redundancy for me.

65813• 15 days ago

USB/SATA adapters are incredibly flaky. Some don't support TRIM, some have major firmware bugs, many overheat during long data transfers and disconnect from the host. I'd strongly advise against using them for any important data.

igoose12• 15 days ago

10 months ago I've built a Raspberry Pi 5 mini NAS with 3 SSDs connected through RADXA Penta Hat. It runs OpenMediaVault, system is on the SDcard, other data are on the SSD ZFS pool. I have autosnapshots enabled, and I tune compression methods on specific datasets (lz4 for almost everything, zstd for file backups).

We use it for storing (backups, media files), playing (PaperMC), and watching (Jellyfin). I can only complain about the lack of hardware decoding in Raspberry Pi 5. Jellyfin loads CPU much if I enable transcoding so it's always disabled. If I knew this, I'd consider a cheaper and faster, but less popular, RADXA machine. Storage is fast enough for me, rsync and Samba speeds are usually limited by my network. PaperMC also runs without a hitch, thanks for asking!

As I didn't have high requirements for the machine, I also considered USB Bays at first but they wouldn't go well with ZFS.

ahepp2• 15 days ago

I'd probably just buy a cheap prebuilt PC with a big case, or surplus enterprise tower. You can get cheap LSI PCI cards and do miniSAS to SATA if needed.

The biggest concern I'd have with USB is power delivery to the hard drives, but I haven't even done the napkin math so maybe it's fine. The SSDs seem like they might be a waste of money. USB hard drives have a poor reputation, but I don't have a ton of experience to say how much of that is deserved. On a practical level, I'd also be concerned about knocking the cable out.

bobmcnamara4• 15 days ago

The big USB racks all have separate power.

turpentine2• 14 days ago

You can get cheap LSI PCI cards and do miniSAS to SATA if needed.

People say those generate a ton of heat because they're normally in a rack mounted server with plenty of airflow. Probably a significant drawback for a "cheap prebuilt PC".

ocd5• 15 days ago

I'm sure people have reasons for taking these things as far as they do with ZFS, and everything else that goes into what is commonly considered a "NAS." But I've found great convenience in a tight NFSv3 config running from a single high capacity HDD with ext4 on a Linux machine (primary system that's always on anyway while I'm around/awake) making things available to my other devices for the electricity and overhead cost of one machine instead of multiple.

I try to keep my network configuration restrictive by default, so I'm not too concerned about possible security arguments running it from my main machine. I've probably committed some great sin here, but is plain NFSv3 and secondarily Samba (for compatibility) really not enough?

dannyw3• 15 days ago

First: Do you have backups of your single high-capacity HDD? That's my biggest worry. What's your plan for when that HDD fails one day, as it will?

The main reasons people go for ZFS and a "NAS" is checksums and data integrity protection, as well as maybe not wanting to keep their primary machine awake all the time (I personally don't).

Then there are useful features like snapshots, which means I don't have to worry as much about accidentally deleting, or over-writing a file and losing data.

I don't see anything wrong with using a main machine that's up 7x24 as a NAS, don't buy things for the sake of something, but I'm worried about your reliability and bitrot protection. (Yes, it happens, I've seen it first-hand thanks to ZFS).

ocd5• 15 days ago

I have external parity for everything with rsync. I don't have any storage that isn't matched with external backups. Fortunately I've never seen any bitrot or had a drive fail on me since I tend to replace them before they do. And yeah, I could add in some extra HDDs for RAID, but this is more a personal choice because of how much I hate SATA.

Filesystems like ext4 are because I value boring and rock solid stability over semi-experimental status of modern filesystems with all the features.

CTDOCodebases3• 15 days ago

It's more the single drive storage layout that is an issue in your case. No resilience to a single disk failing and you are risking all your data if your hardware starts silently corrupting data. This is assuming all your backups are done using the data on the single hard drive.

If you want to keep your setup simple I would consider the following:

- Keep a list of file hashes and check your files against this list periodically.

- Use some type of backup program that supports snapshots. Restic is a good choice.

- Get a hard drive the same size as the one you have and use SnapRAID to manage file integrity or set up a two disk mirror using btrfs.

zymhan• 15 days ago

Once you add multiple machines reading and writing simultaneously, you'll hit a limit. But that's fine, build what you need now, no need to over-optimize.

a-french-anon• 15 days ago

I think it's appropriate to link to my blog post about building such a NAS but with very different technologies: dm-integrity, mdadm and XFS. The reason being that as a C/C++ dev following OpenZFS development closely, I found the focus on ginormous features over general stability and the constant problems due to the SPL/separate page cache too worrying.

Sure, I traded the convenience of kitchen sink (or Swiss army knife for the more charitable souls) ZFS for some initial pain, but I'm very happy with my choice today.

https://world-playground-deceit.net/blog/2025/06/nas-setup-l...

dannyw3• 15 days ago

A friendly reminder since it's probably relevant: when's the last time you tested disaster recovery for your own setup? If you haven't verified recovering it, you don't have an offsite backup.

I had a cloud backup (Backblaze B2, using rclone and encryption) for my home NAS. Some unlucky drive failures later, I was getting ready to recover from my Backblaze backup... and then I couldn't find/remember where I saved the rclone encryption backup.

I lost all data in that NAS, including irreplaceable personal/family photos, due to that mistake. My lesson to share: please verify you can indeed recover from backups.

PeterStuer2• 15 days ago

Rather than 'plain' debian you can put it on Proxmox, and enjoy the bliss of Proxmox Backup Server.

iJohnDoe• 15 days ago

Can you explain a bit more about this. Would this help in a single server to facilitate backups? Or does this need the the NAS and the Proxmox Backup Server on a different system?

PeterStuer2• 13 days ago

I'm running PBS on a separate mini pc, it is the perfect forever near instant incremental backups that sold me. I did perfectly recover in 20 minutes after a full hardware failiure. Best recovery experience ever.

FireInsight• 15 days ago

I used this blog post as one learning resource while building my own NAS a while back: https://xyny.art/blog/2026-building-nas/

Building a NAS from scratch is really fun! A small hinderance, but definitely part of the fun as well, is the lack of a "complete resource" on the topic covering how to do every single thing you need to do. Part of the point of my blog post is actually to bring to the internet yet another opinionated NAS setup "guide" (eventhough I would hesitate to call it a guide, but if I ever had to do the same thing again I would definitely read my own post first).

devn0ll• 15 days ago

I made 2 Raspberry Pi5 NAS's. one with 4x 6TB and one with 4x 12TB.

Used mergerfs and snapraid, and a simple NFS share. Absolutely perfect for Proxmox backups, our pictures, media, etc, etc. No fuss, easy replacement of drives without needing to keep drives of the same size around.

I'm good with ZFS, have been for years now managing storage for $COMPANY. And I still freaking love using zfs send/receive with proxmox ;-)

But for most at home stuff: mergerfs and snapraid are just more logical.

deeddy• 15 days ago

To be totally honest, I don't think I can live without ZFS anymore. Deduplication and snaphots in 2026 are a must.

Maledictus• 15 days ago

and checksums!

MrDOS2• 15 days ago

Some other things I find useful on NASes:

Install avahi-daemon. Samba will automatically register with it to advertise SMB/CIFS to macOS and Linux clients over DNS-SD.

Install wsdd2 so that your server will be auto-discovered by Explorer on Windows 10+ clients with SMB 1.0 disabled, too.

Your Linux hostname is probably lower-case, but by default, Samba publishes a capitalized rendering of the hostname to NetBIOS and Avahi. If this bothers you, set “host-name=something” in the [server] section of /etc/avahi/avahi-daemon.conf, and set “mdns name = mdns” in the [global] section of /etc/samba/smb.conf.

If you have macOS clients, you should enable vfs_fruit in your Samba configuration: https://www.samba.org/samba/docs/current/man-html/vfs_fruit..... There are some compatibility reasons to do this, but mostly it means you can set the “fruit:model” so that your server has a fun icon in the Finder sidebar.

To avoid the creation .DS_Store files, you can disallow them: https://ryanoberto.github.io/blog/2015/04/01/disabling-the-c.... I think you can also set “fruit:resource = xattr” to store Finder preferences in xattrs, but I haven't tried it.

Although macOS deprecated AFP in favour of SMB years ago (and are slated to remove AFP client support altogether in the upcoming macOS 27), SMB client support in macOS is still pretty miserable. The upcoming macOS 27 is set to drop AFP support, but until then I will continue to run Netatalk side-by-side with Samba. Netatalk also registers itself with Avahi, and macOS will (tellingly) use AFP preferentially to SMB, so clients will talk to the right daemon automatically.

xattt• 15 days ago

For Samba, add a separate entry for the user home directory because the homes directory default triggers some sort of OCD.

MrDOS2• 15 days ago

Yeah, the default [homes] share is really not necessary on a home network where you really just want a common share. On my home storage server, I just have a “files” user, and a corresponding, anonymously readable/writable share:

  [public]
     vfs objects = catia fruit streams_xattr
     fruit:encoding = native
  
     comment = Public share
     path = /home/files
     force user = files
     force group = files
     guest ok = yes
     browseable = yes
     read only = no
     create mask = 0644
     directory mask = 0755
  
     readdir_attr:aapl_max_access = no

The default [homes] share is still there in case I need it for anything, but it's “browseable = no” so it doesn't confuse visitors.

Beijinger• 15 days ago

Has anybody used unraid for his Home Server? Is it worth it?

unraid.net

joshmn• 15 days ago

I'm hardly a sysadmin and I think Unraid is great. My usecase is some Docker containers and media backups. The UI gets the job done well, whereas Synology tries to dumb things down and make things pretty.

Android is to Unraid as iPhone is to Synology.

sleepycatgirl• 15 days ago

personally, after buying X380, using my old T430 as a mini NAS..(also immich) with a ZFS mirror. works wonderfully :3

rcarmo• 15 days ago

This is pretty much what I did, except that since I use Proxmox I used that to manage the volumes and then just did a pass-through mount to an LXC that runs Samba - that way I get a decent, simple GUI to monitor and manage volumes and can do user management inside the LXC.

sgt4• 15 days ago

My homelab "NAS" is unfortunately just a few USB based external disk enclosures, externally powered. Is there no way of making these reliable ? Every so often (maybe once a month) I get timeout errors or some kind of issues with at least one of the disks.

a963• 13 days ago

Apart from better (luck with) enclosures, cables, and power, probably not. USB-SATA bridges are notoriously flaky.

turpentine2• 13 days ago

Try a different host machine to rule out it being a DAS enclosure.

thisisauserid• 15 days ago

Needs a 2024 in the title maybe.

Also, are there neal.* besides .fun and .computer?

I'll check all the other TLDs real quick...

rmoriz• 15 days ago

Given the current "crisis", I wonder if older off-the-shelf NAS from QNAP, Synology, Asus, UGreen etc. could be re-flashed using FOSS.

whalesalad• 15 days ago

A lot of them can, yes.

a963• 13 days ago

The models that are a crappy PC can, almost like any or other crappy PC. Some have a very tiny DOM that might or might not be enough for a recent system. Of course, once the system is minimally up, it can use the big disk for software. No idea if any have special weaseling on them, but you can always search for the model and e.g. Debian (that tends to have best hits). Running with read-only root or just initrd is also quite possible.

The ARM models tend to be very very very slow and small and have architectures that are obsoleted in many Linux distributions combined with hardware that's weird enough that NetBSD doesn't support it (e.g. LAN chip or SATA, which you might need in a NAS). And yes, looking at NetBSD pages and build configs is one avenue for finding well supported devices. Although the documentation isn't great or complete. Some models may also have weird boot sequences or rescue modes that are extra hassle.

I keep wondering if a neat little buildroot config would work with some of these devices. Other odd options would be https://armbian.com/boards and https://openwrt.org/supported_devices .

j1elo2• 15 days ago

I'd love reading some blogs about what's the best usage for actually minimal computers, any suggestions?

It always gets me how the world of self-hosting is usually introduced from claiming that you can start giving a second life to a Raspberry Pi or a forgotten laptop, and suddenly the next blog you read calls "minimal" a beast machine meant for racks and semi- or professional environments.

Bought a ThinkCentre M910q with an internal SSD and 16 GB of RAM for €200 a couple years ago... Right now I got it chugging along with TrueNAS + 2 USB disks in ZFS Mirror (sitting in a closed cupboard so no chance of cable disconnections).

For me, "minimal home server" means a small computer that fits on a cabinet on the living room, is practically silent, and has a very small power consumption profile (less than a decent Hetzner otherwise the cost wouldn't be worth it). I have a mini-PC in mind, but people think of Dell PowerEdges. Even if given for free, I would never install at my home a PowerEdge for a home server.

I guess it must be the difference between living on a flat vs a 2 story house :-)

dwedge2• 15 days ago

ThinkCentre M910q

I love these machines. I have 3 of them and an M720Q. Needs changed so only one of each is actually in use, but they are so low profile and quiet that I'm considering deploying the other two at home just in case I ever get the urge to play with K8s or something. The only thing I'm missing is a splitter so I can use one plug socket instead of three - not sure if something like that exists.

One nice little fact about the M910Q is that they are rated to maximum 32GB but I have two of them with 2x32GB no problem. Recent RAM prices and forever regret is the only reason the third one doesn't have the same. The M720Q only takes 32GB though.

You can also run 2 SATA off of the M2, and someone created a 3D printable enclosure for them: https://makerworld.com/en/models/1280680-thinknas-2x-hdd-enc...

j1elo2• 15 days ago

So it has the potential to even look and work like an actual NAS! That is so cool. Sad state of affairs that 64GB seem to be around €400 nowadays, that's actually double of what the whole machine cost. I'll wait for the AIpocalypse and come out later to buy some sticks.

kjs3• 15 days ago

I run conceptually the same thing, but on FreeBSD instead of Debian. Different set of trade offs; not per se better or worse, just different, and in the end works just as well.

VikingCoder• 15 days ago

Where can I have a discussion with other developers about storage?

Is here okay?

I want to have physical storage over here, and logical storage over there, and I want to control the mapping from one to the other. I want to talk about encryption, replication, latency. Make "time machine" available on this logical storage. And then I buy some new physical storage, and it joins the story. This physical storage is over at my friend's house for backup. This physical storage is slow and archival. This is a device for writing archival media, and there's a brand new media in it, go ahead and write to it. Show me the health report on all of the physical media, and show me what you've done to protect the logical storage. Graph my usage and make suggestions about when to add more physical storage.

Buying physical storage with power and wifi, and configuring it with a QR code that's on an e-ink display - seems like it should be the most obvious thing in the world, that we should all be really used to doing by now.

What am I missing?

koverstreet• 15 days ago

bcachefs is slowly working up to what you're describing - describing what you want and letting the filesystem sort it out. We're basically there for local storage, and other people have been building some nice reporting on top.

Next year (post Rust, because networking code is so much nicer in Rust) will be send/recv, and I think we should be able to make some nice improvements over the state of the art there.

shark12• 15 days ago

If you use LVM you can have a similar pool of storage with 1GB RAM or even less, instead 8GB (ZFS requirement).

Caveat: LVM does not have built-in snapshots like ZFS has.

regexorcist3• 15 days ago

LVM does have snapshots.

solarengineer3• 15 days ago

"It is important to regularly monitor the snapshot’s storage usage. If a snapshot reaches 100% of its allocated space, it will become invalid." [1]

ZFS snapshots use Copy-on-Write differently and have no such allocated space limit. Thus, we can do interesting things like snapshotting a file system after an OS is installed, and then roll back to that snapshot upon an OS upgrade failure, or even clone a new file system from that snapshot to have a different copy of the OS.

Essentially, the nature of the snapshots are different in both.

[1] https://docs.redhat.com/en/documentation/red_hat_enterprise_...

impact1232• 15 days ago

Where does that 8GB requirement come from?

shark12• 14 days ago

From the official documentation.

vardalab2• 15 days ago

That's what I did after my 15 year old Synology DS1010+ built in USB DOM failed. Put a network booted debian on it via netboot.xyz with zfs and now i get to reuse those 15 year old 2TB still chugging along disks. Fully open source, pretty nice way to keep old hardware going. It's my tertiary backup that wakes up once a week pulls in some open weights LLM models that I hoard just in case and goes back to sleep.

mrighele• 15 days ago

Having built a nas recently (luckily before ram/diskpocalypse) there seems to be a huge difference between making a small nas (say 4 disks) and a big one. With a small one you are essentially building a pc, and the differentiating factor is the software. Mobo, memory, case, PSU… buy consumer (cheap) stuff and you’re ok. Above that you have to start buying very expensive business oriented stuff,disk enclosure, better psu… the costs increase more that linearly , especially if you don’t live in the US and can easily get used stuff from eBay.

And that stuff is often not just more expensive, but uglier and noisier. I ended up making my own “enclosure”

dpedu• 14 days ago

I've been re-building my NAS and decided on NFS on top of ZFS. However, my goals are slightly different than the author: I'm using a low-power cpu/motherboard, and I've built 2 distinct systems that can fail over as a form of high availability. I've actually written some custom software to handle the failover and data replication. In my testing, clients see writes block for just a couple second during failover. Would anyone be interested in a writeup about this?

How to Build a Minimal ZFS NAS Without Synology, QNAP, TrueNAS (2024)

Related Stories