A straight talk about btrfs

https://www.suse.com/communities/blog/butter-bei-die-fische/

by cyphar • 9 years ago

204 162 9 years ago

suse.com

btilly• 9 years ago

For those who wonder who deprecated it, Red Hat did in the release notes for 7.4.

Search for brtfs in https://access.redhat.com/documentation/en-US/Red_Hat_Enterp... for details.

lemoncucumber• 9 years ago

As has been discussed in other threads, RHEL's kernels are ancient compared to mainline and so they probably decided it's just too much work to continually backport btrfs changes onto increasingly dated kernels. The sky is not (necessarily) falling.

jameskegel• 9 years ago

What put t in perspective for me was that here's a company who was a product and they make their money giving those products the utmost attention to things like compatibility and stability, and that means helping when these things break. This is not them making a political move or a line in the sand, this is them saying they can not afford to put the same 110% into this project so they're not going to carry it forward. It's the circle of life and I appreciate them for the rest of their works even if it can be mildly inconvenient in some cases.

microtonal2• 9 years ago

But if they thought that btrfs was the future, they would definitely have put in the manpower to maintain and backport btrfs. Red Hat probably believes that XFS + LVM et al. can provide the necessary functionality for enterprise setups on a more reliable basis.

greglindahl• 9 years ago

Uh, the entire value proposition of RHEL is that Red Hat will backport such patches, supporting customers who don't want to change anything until they move up to the next major release.

djsumdog2• 9 years ago

Looks like they're running 3.10 on RHEL7.4, which is a pretty old long term release. I mean there are long term 4.x branches now.

j_koreth• 9 years ago

Is there any reason why Oracle contributes so much to Btrfs while maintaining ZFS? Or am I mistaken in Oracle's relationships with ZFS

riffraff3• 9 years ago

oracle has its own linux which it is interested in moving forward, but its version of ZFS is restricted to Solaris.

Elv13• 9 years ago

Oracle sells Oracle Linux, a RedHat clone. ZFS on Linux (ZoL) isn't controlled by Oracle, it is a fork of OpenSolaris version of ZFS. The ZFS sold by Oracle on Solaris isn't the same product and doesn't support Linux. Oracle doesn't want to support it on Linux because it would fragment one of the remaining cash cow they got from Sun. And anyway, they probably can't support ZoL even if they wanted to without putting themselves into self induced legal gray water. Apparently, they intend to move Solaris into a rolling model like Windows 10, but focused on legacy. They can sell ZFS there without affecting their Linux operations. There is also a rumor of a deal with NetApp not to support ZFS on Linux. All combined, BTRFS is a better suited tech for them to support.

rehemiau• 9 years ago

I was wondering, how does Illumos fit into all this? Is their ZFS the Oracle-controlled one?

boomboomsubban5• 9 years ago

They started btrfs several years before they acquired Sun and ZFS with it. Why they continued with it is unclear, presumably a combination of inertia and not wanting to relicense it for other Linux's to be able to ship with.

perlgeek• 9 years ago

This is a bit of a tangent, but I think it's important to remember that a huge corporation like Oracle has quite a different decision making process than individuals.

Yes, one devision of Oracle owns, maintains and develops ZFS. But one of the many other divisions (maybe for historical reasons, maybe because it was acquired and never migrated, ...) might use btrfs, and it makes sense for them to pour manpower into it, even if that could be perceived as somewhat of a competition to ZFS.

My main point is that big organizations naturally tend to do things that look conflicting from the outside, just because they are too large to be efficiently standardized.

_pmf_• 9 years ago

This is a bit of a tangent, but I think it's important to remember that a huge corporation like Oracle has quite a different decision making process than individuals.

I'll just leave this here for people to watch: https://www.youtube.com/watch?v=-zRN7XLCRhc

bitL• 9 years ago

Oracle started BTRFS as "me too" project to show SUN they can do something like ZFS (well, they couldn't). So after acquisition of SUN they ended up with both. SUN's strategy was to use ZFS as one of main selling points of Solaris and refused to port it to Linux; somebody someday decided to port it as well and ended up with ZFS with weird license and buggy BTRFS they slowly phased out to Red Hat and SuSE.

giis• 9 years ago

Recent Redhat decision may also be related to their investment in Storage products Ceph (~175 million in 2014) and GlusterFS (~125million in 2011) not just about stability of btrfs.

As someone who previously experienced with distributed storage like emc-Isilon and also fuse-based glusterfs, btrfs has huge potential for enterprise storage. It does also need more effort on testing front at this moment. hoping that soon btrfs will become default fs for most Linux distros.

ghaff• 9 years ago

What do distributed software-defined storage/distributed filesystem projects have to do with a CoW filesystem? Alternatives to btrfs are things like XFS and ext4.

jle17• 9 years ago

I read somewhere (probably on HN) that it could be related to their acquisition of Permabit, a company which seems to be producing Linux software for deduplication, compression and thin-provisionning. This seems more in-line with what btrfs has to offer.

microtonal2• 9 years ago

If Brazil, one of the world biggest producers of beef, would announce to stop producing fish: would you wonder, whether Peru, one of the world biggest producers of fish, would stop producing fish?

I find this quite weak argumentation coming from SUSE. Novell/SUSE was also (by far) the largest contributor to AppArmor at some point. And SUSE used it as the MAC framework in their consumer and enterprise distributions. Then seemingly out of nothing they fired the AppArmor team.

jabl• 9 years ago

Oh? What are they using now then? SELinux?

sp3322• 9 years ago

AppArmor is configured to run by default on any fresh installation of SUSE Linux Enterprise Server. https://www.suse.com/documentation/sles11/book_security/data...

All the drama was back in 2007. https://web.archive.org/web/20160410165126/http://www.cnet.c... Looks like it blew over by 2008 https://news.opensuse.org/2008/08/20/opensuse-to-add-selinux...

cyphar13• 9 years ago

Both openSUSE and SLE use AppArmor by default. I believe we provide some support for SELinux if you want to use it, but AppArmor is the default LSM.

appleflaxen2• 9 years ago

that's an important fact about the sustainability of any corporate contributor, but it's not relevant to the thesis of the article (which is that any corporate contributor makes decisions independently than others).

rodgerd3• 9 years ago

That's gotta be the weasel-wordiest straight talk I've seen in a wee while.

Here's a fact: the upstream's official statement is that RAID 1 isn't production-ready.

cyphar13• 9 years ago

That's not accurate. Upstream btrfs doesn't support RAID56[1]. However, neither their multiple devices page[2] nor their gotchas page[3] mention that raid0, raid1 or raid10 are not recommended for production. Do you have a citation for your claim?

[1] https://btrfs.wiki.kernel.org/index.php/RAID56 [2]: https://btrfs.wiki.kernel.org/index.php/Using_Btrfs_with_Mul... [3]: https://btrfs.wiki.kernel.org/index.php/Gotchas

rodgerd3• 9 years ago

https://btrfs.wiki.kernel.org/index.php/Status

"RAID1 mostly OK Can get stuck in irreversible read-only mode if only one device is present."

Which is an especially toxic combination with things like:

"Device replace mostly OK"

It's 30 seconds to find the status page if you're actually trying.

wtallis14• 9 years ago

"Here's a fact: the upstream's official statement is that RAID 1 isn't production-ready."

That doesn't seem to match anything I've seen. The btrfs status page classifies RAID1 and RAID10 features as "mostly OK". The one and only documented caveat is that when a disk in a RAID1 fails, you should mount the filesystem as read-only until you are ready to fix the problem by either replacing the failed disk or converting to a non-RAID profile. There's a real bug underlying this limitation, but the only way to encounter this bug is to be quite cavalier about how you handle a degraded array, by doing something that you shouldn't expect to be safe.

rodgerd3• 9 years ago

but the only way to encounter this bug is to be quite cavalier about how you handle a degraded array, by doing something that you shouldn't expect to be safe.

The way to encounter it is to mount the array a second time. That's hardly "cavalier handling".

A decade in, and RAID 1 doesn't work right. Saying this clearly offends the feels here at HN, but it's a fact.

michaelmrose• 9 years ago

Mostly OK != production ready.

A degraded array should have no substantially greater risk of failure than the single disks the vast majority of people rely on daily. I can think of one obvious reason one might mount the degraded array simply to copy data from it to a different arrangement. Another reason is in a non enterprise use case a desktop users might not instantly have a disk available and might mistakenly believe that this is no more risky than having only one disk in the first place.

This is ridiculously flaky for something that is supposed to be years in development with backing from major companies.

This confirms my feeling that btrfs belongs in the waste basket.

mulle_nat3• 9 years ago

Ran into this with my RAID1: https://www.spinics.net/lists/linux-btrfs/msg62346.html. I then ditched btrfs, and I'll never try it out again.

linuxready4• 9 years ago

Do you have a link to this real bug ? It would be interesting to follow it until it is resolved.

jacknews2• 9 years ago

A data-point of one, and I only have a few TB spread over a handful of disks, but I lost an entire 1TB btrfs volume recently because of a single disk sector gone bad (of course, I have backups, but it's a pain to restore 1TB, not to mention racking of the nerves (will the backup drive fail with sudden heavy use?)

It's the only FS I've ever been unable to recover data from, at all, (I've had quite a few disks with errors before going back 25 years), and so it's no longer on my list of filesystems-I-trust.

andrewstuart2• 9 years ago

I've experienced data loss twice in 15 years of using Linux and trying different file systems. Both were on btrfs. I didn't use any advanced features, though I had plans to before this happened. But at that point, I cut my losses and converted all my disks back to ext4.

The features are great, but data availability/integrity is the fundamental thing I want from a filesystem. If I write some bytes, they should be there tomorrow. Everything else is secondary.

cyphar13• 9 years ago

If you're interested in a more featureful filesystem that isn't btrfs and has a very solid track record, I would recommend looking at XFS. It's a very old filesystem but has a lot of quite modern features (and it performs better than ext4 for several workloads).

aljarry• 9 years ago

I've had similar experience around 5-6 years ago. One of the notebooks with ubuntu was force rebooted and btrfs got corrupted, no way to boot the system again.

vog3• 9 years ago

Serious question: Why would anyone today use BTRFS, given that the battle-proof ZFS has finnally been ported to Linux for quite some time?

josteink2• 9 years ago

Why would anyone today use BTRFS, given that the battle-proof ZFS has finnally been ported to Linux for quite some time?

It's only been available out of the box in a simple-to-use and supported fashion in 1 Distro for barely 1 year.

You can hardly say it's been widely available, nor for a "quite some time".

mulle_nat3• 9 years ago

Because I want my machine to do something else besides running a filesystem. :P

intsunny• 9 years ago

Some of us are very concerned about licensing at all times. I'd wager such steadfast concern is ultimately what helped Linux in matters of SCO v. IBM over the matters of IBM's JFS file system.

ZFS is very much in a grey area, and in turn, a huge turn off for many.

pgeorgi• 9 years ago

At that point, why rely on Linux?

pantalaimon• 9 years ago

The possibility to dynamically add and remove discs from the array is a pretty neat feature.

moogly2• 9 years ago

It's the default fs on Synology NAS:es.

user5994461• 9 years ago

>> given that the battle-proof ZFS has finnally been ported to Linux for quite some time?

ZFS has not been ported to Linux yet.

At best, one could say that there are SOME pieces of it being ported and tested on -exclusively- the latest Ubuntu.

JdeBP• 9 years ago

Hacker News recently had a discussion of this at https://news.ycombinator.com/item?id=15011181 .

xioxox2• 9 years ago

BTRFS has much more flexible snapshots and clones than ZFS. You can create rewritable snapshots and create new snapshots based on those. In addition you can create COW copies of files with "cp --reflink" which ZFS doesn't support.

BTRFS feels much more like a native filesystem on Linux too. It doesn't have that big ARC cache.

We've been using BTRFS for 5+ yrs on a Linux MD RAID setup, with no problems at all.

linsomniac3• 9 years ago

I love ZFS, but I'll also say that I also feel like ZFS is fairly slow. Of course, it probably doesn't help that most of my ZFS machines hold 10+TB of data, with lots of snapshots.

I don't feel like I'm going to be using btrfs anytime soon, I've given up on it. But there are days I wish I had an alternative for high reliability, snapshotting, and ideally deduplication (which usually tends to make my ZFS machines fall over).

I'd love to see HAMMER on Linux.

tqh• 9 years ago

Same here. I lost a disk to btrfs and did not manage to recover any data. No longer on my list of filesystems-I-trust. I guess Suse is the biggest contributor since the others stopped? It doesn't give me any confidence that it can be trusted.

RX142• 9 years ago

Which raid level were you using over those multiple disks? I've been using btrfs and never even experienced a hiccup despite continuous sector failures, continuous drive failures and even disk controllers disconnecting on the fly. And I've been running the whole thing on a single, fairly weak, server with second-hand disks attached over the cheapest USB3 sata adapters I've found. By all accounts I've been putting btrfs in a situation that is well outside the norm and I have 0 complaints. I also use it on my desktop (no raid) with similar experience (minus the shitty hardware).

I am continuously amazed at the amount of people who have issues with btrfs. It's been absolutely rock solid over the time I've used it and I have 0 complaints (apart from the ext4 -> btrfs converter producing a corrupted btrfs filesystem, but the actual kernel btrfs code itself has been flawless).

btmaybe3• 9 years ago

I too have lost an entire btrfs volume (single disk) because of a bad sector. It seems btrfs is very sensitive to bad sectors in metadata, compared to ext4.

Since then, I've formatted my btrfs partition to do RAID1 on metadata and haven't had problems since.

  $ sudo btrfs fi df /
  Data, single
  System, DUP
  Metadata, DUP

atomi• 9 years ago

I have the same experience. btrfs has been great to me. I've disconnected disks on the fly as well on raid 1 and the mount is happily serving files off the remaining disks.

grumpyprole• 9 years ago

I'm really confused by the choice of development process for BTRFS. First they write huge amounts of experimental/buggy code and then spend years trying to fix it. Reliability was obviously not the primary goal and I doubt they will be able to retrofit it.

kingwill101• 9 years ago

2 days ago I shutdown my laptop and all was working out great at that point. Now yesterday I turned back on to get some work done and couldn't pass maintenance mode because my home partition got corrupted somehow. Took me all day literally to get it working. I have since changed that partition back to ext4 leaving only / as btrfs

unixhero• 9 years ago

I use BTRFS for non critical data, like scratch data and second backup archives. This is because I can stretch it out over many old harddrives I have laying around. The memory footprint is also very low. And it comes with compression!

But for everything else, I have rammed up, manned up, and use ZFS.

memracom• 9 years ago

This is a bad move. More than ever we need Linux standards and this goes in the opposite direction. It would be OK for SUSE to support btrfs as a first class choice of filesystem but the extfs family should be the standard one.

And ZFS works just fine on Linux. If people want to use it, then the distros should not put roadblocks in their way and that means, ZFS should also be supported as a first class choice for servers.

Note that "support" is different from licensing and from "included in our install repo". It is OK to have different licensing for things like this and to install it from a non-distro repo. Look at PostgreSQL for an example of how you can install a mission critical tool from a distro-compatible repo that is run by the upstream project, in this case, PostgreSQL.

But even though it comes from a different repo, it should be "supported" by the distro to the extent that they make a best effort to help people resolve problems. It doesn't mean that you need to be experts in every nuance of the tool and the best way to do that is to maintain a good working relationship with these upstream projects.

Something like ZFS or PostgreSQL are mission critical tools that use Linux as the interface to the hardware, My comments do not apply to any random app or utility that someone wrote for Linux. Perhaps btrfs belongs in this class but I personally don't know since I have not used it.

gens2• 9 years ago

Linux has lately gotten a lot of bad standards.

iso-8859-1• 9 years ago

Such as?

wongarsu• 9 years ago

More than ever we need Linux standards

Why is that? Given sufficient resources, isn't competition and innovation better?

dom0• 9 years ago

but the extfs family should be the standard one.

*XFS

gbrown_• 9 years ago

TL;DR - Don't be dissuaded by marketing, here's some more marketing.

Elrac• 9 years ago

When a CEO/CTO uses the words "straight talk" I prepare for a copious load of meaningless waffling. Was not disappointed!

appleflaxen2• 9 years ago

huh. i found it to be coherent and not-particularly-heavy on techno-doublespeak.

while it wasn't particularly edgy/straight, it certainly wasn't "meaningless waffling".

i got the impression the title came from wanting to use the german idiomatic expression for "straight talk" in order to work in the butter-cow pun.

hyperfekt• 9 years ago

I've taken a long look at COW filesystems in these past days as part of me wanting to set up a new workstation, and it appears that while btrfs is not a viable long-term solution for its quality issues and lack of development, ZFS does not have the pretense of being a filesystem for the general usecase, as exemplified by the lack of offline deduplication, defragmentation, the possibility to easily change disk configurations, and much more. Maybe we should consider accelerating the development of bcachefs as the future of reliable and feature-rich filesystems on Linux, which appears both more modern and holistic, but still has quite some ways to go. A new filesystem is necessary because many things we should demand from them are not modularly composable without massive disadvantages. Implementing compression as a layer, for example, demands basically creating another filesystem to manage the space, with great overhead in all dimensions. Similar things go for the consistency guarantees provided by COW or the deduplication and snapshotting that depends on it.

A straight talk about btrfs

Related Stories