BSDs or Illumos may be good alternatives to Linux (2017)

https://www.adminbyaccident.com/politics/abandon-linux-move-freebsd-illumos/

by O_H_E • 8 years ago

69 116 8 years ago

earenndil• 8 years ago

PS: I haven’t mentioned both softwares, FreeBSD and SmartOS do have a Linux translation layer. This means you can run Linux binaries on them and the program won’t cough at all

I don't know about SmartOS, but on freebsd, linux compat is at version 2.6.38, and so most programs don't work.

equalunique2• 8 years ago

Is that for ports of linux software, or for "Triton" images?

pdx62• 8 years ago

SmartOS will run Linux distros in an LX zone, much like Solaris did. There are some limits around what syscalls are available, but generally most software works or a bug can be opened. There is support for Linux containers natively, but the last I checked only Docker v1 repos were supported. v2 is forth coming. Virtualization-wise, Bhyve is supported as is KVM.

If you are running open source, there are usually SmartOS packages in pkg src, which makes SmartOS zones a CPU/memory/security bargain. https://pkgsrc.joyent.com/

RantyDave• 8 years ago

I haven't run SmartOS for about a year, but I was using it to run Linux binaries. It was fundamentally great. Stable, efficient ... great. Joyent were nailing compatibility bugs in a matter of days.

A couple of small things - one is that entire distributions "just as they are" are unlikely to work, in much the same way as trying to boot a docker container doesn't. Lots of things rely on /proc and cgroups and these are big, ugly interfaces that are never going to see parity between the two OS's. /proc is (amusingly) built as a r/o filesystem that has an htop-esque thing writing state onto it constantly.

The big one is memory allocation. Linux will let memory allocations fail. It will also just shoot apps in the head if it wants to. Solaris does not do such shit and will return the malloc call when it has allocated some memory - even if it's next week some time. On more than one occasion I have been completely locked out of a SmartOS box because of this and one ends up having to be really, really careful with allocation strategies as a result.

craftyguy• 8 years ago

So the main points of this rant are:

- BSD has ZFS (but so do some Linux distros)

- Linux bad because BTRFS, so it's bad (well, you don't have to use it)

- BSD jails > anything on linux (I was hoping to see a clear outline of pros/cons, but the author's reasoning is very hard to follow, and the tone is very combative instead of constructive)

hacknat2• 8 years ago

At this point there really isn’t any security difference between zones, jails, and hardened containers. That word “hardened” is the key. Zones are a primitive in Illumos, containers are not a primitive in Linux, they are a collection of security features. If you use them all then you can achieve parity with Zones, but surprisingly few projects make use of all of them by default (Docker certainly doesn’t).

The main difference then is usability, which is almost always the biggest security flaw in any piece of software (that the security isn’t “on” by default or has a prohibitive learning curve for the user).

posix_me_less4• 8 years ago

Docker certainly doesn’t

Do you think LXD gets closer to Zones in terms of security?

qlk1123• 8 years ago

Just a weakly related comment here.

When one talks about security, it's really hard for people to follow what she really means; the context has always been qualitative, but not quantitative.

I found the idea "Vertical/Horizontal Attack Profile" interesting, which may help measure security when people argues different container solutions or even VMs.

https://blog.hansenpartnership.com/measuring-the-horizontal-...

pdx62• 8 years ago

SmartOS has native ZFS support and zones.

rbanffy4• 8 years ago

Isn't SmartOS and Illumos essentially the same OS?

lmm• 8 years ago

- Linux bad because BTRFS, so it's bad (well, you don't have to use it)

The point is that there are no good filesystem options for Linux. ext4 has various issues mainly due to being ext2 underneath, reiserfs is unmaintained, xfs and jfs have no recovery tools at all (and xfs additionally has the trailing zeroes problem), and all of those require you to use a separate system for RAID for which Linux has no good options (true hardware RAID is expensive, BIOS RAID (dm-raid) and software RAID (md) will kick out disks during a rebuild if they see a single bad sector, which per https://www.zdnet.com/article/why-raid-6-stops-working-in-20... is more likely than not with modern disk sizes. This risk is reduced (but not eliminated) by setting up regular scrubs, but tutorials and installation wizards don't always do that). BTRFS has issues up to and including data loss, userspace ZFS has major performance and reliability issues while in-kernel ZFS on Linux has substantial licensing concerns.

- BSD jails > anything on linux (I was hoping to see a clear outline of pros/cons, but the author's reasoning is very hard to follow, and the tone is very combative instead of constructive)

Agreed that the writing is bad, but BSD jails do have a much higher level of maturity than linux containers. Linux is probably catching up now given the sheer weight of numbers of people using containers, but I still wouldn't trust them quite as much as BSD jails just yet; various kinds of flakiness and security vulnerabilities still seem relatively common.

setr• 8 years ago

I'm naturally inclined towards BSD simply because of its extensive history (though I've spent little effort towards it thus far), but even still this article is wholly unconvincing... of anything. It really doesn't say anything about ZFS, jails, or even the current state of affairs beyond a respectable number of packages (and more importantly, linux compatibility); it just assumes you already accept that they're better, and for some reason haven't already switched?

It also badly requires editing

Hopefully someone has a better argument for FreeBSD than this... mess

AHTERIX5000• 8 years ago

Yeah TFA reminds a bit of passionate blog posts and articles about switching from Windows to Linux something like a decade or two ago. Something that would fit better in BSD community such as Lobste.rs than HN ;-)

writepub• 8 years ago

The selection of software that simply works on Linux (especially Debian & derivatives like Ubuntu) eclipses that on FreeBsd.

I've seen attempts at bringing the Linux ABI to FreeBsd, including the familiar and popular apt package manager. If and when it's as easy to "sudo apt-get install package" on FreeBsd as it is on Ubuntu, it will see an increase in adoption in both the sever-production setting, and as a development environment.

compsciphd• 8 years ago

what about debian/kfreebsd (never used it, just wondering)

hackeraccount• 8 years ago

I don't think that's being released anymore.

ComputerGuru3• 8 years ago

I've seen attempts at bringing the Linux ABI to FreeBsd, including the familiar and popular apt package manager.

That's ridiculous. `pkg` on FreeBSD is a thousand times more consistent and user friendly. Perhaps you are talking about the days before pkg-ng (next generation) became the default, but now installing anything is as easy as `pkg install foo` and it just works. It even updates automatically so URLs aren't out of date.

bayindirh15• 8 years ago

I have a deep respect for BSD family, and their way of doing things, but regardless of OS and distro, apt is probably the most advanced package manager currently.

When used with aptitude, it's very easy to use, extremely powerful and flexible. Also, it has the most extensive knowledge on details of packages in terms of dependency graphs and modification of files which belongs to packages.

At least, that's what I saw in 14 years.

rleigh4• 8 years ago

Have you seen the current size of the ports tree? It's on the scale of a distribution the size of Debian.

While there is certainly some Linux-specific software which doesn't work on FreeBSD (and vice versa to a smaller extent), this is a tiny fraction of the total amount of portable free software packaged for both.

rbanffy4• 8 years ago

It's not easy to accidentally make software that compiles on Linux and fails on BSD.

equalunique2• 8 years ago

Project-Fifo uses bhyve, zones, and jails to blend FreeBSD & SmartOS into your own self-hosed IaaS. Worth checking out.

2bitencryption• 8 years ago

Curious, what's Darwin like? Is it much use outside of the Apple/macOS ecosystem?

etaioinshrdlu• 8 years ago

There used to be some projects using Darwin outside of macOS etc, but they are long dead. People have apparently had a harder time even getting the kernel to boot nowadays for various reasons.

pmoriarty• 8 years ago

BSD doesn't have systemd. That's a huge win right there.

bayindirh15• 8 years ago

I also despised having systemd for some time, but it's not giving big headaches in our installations. So I adapted and got used to it.

Besides being a multi-role-somewhat-messy-blob, what are the grief points of systemd in 2018?

Edit: carified the question.

jimpudar• 8 years ago

I'm also wondering this. I've seen plenty of hate for systemd, but I've never personally run into any issues with it.

rawfan• 8 years ago

Check out https://twitter.com/systemdsucks

My biggest personal grievance is how it handles DNS or rather how unpredictable and undocumented the process is. If everything fails, e.g., they actually hard coded usage of 8.8.8.8. Not sure if that’s still there or configurable. The handling of actual security reports and CVEs is rather bad and you can see many examples of that on Github.

I’m not anti systemd and overall I welcome lots of contributions Lennard made in the past (pulseaudio, e.g.). With systemd it’s just too much functionality in one place. Do we really need an NTP alternative in systemd?

jakeogh• 8 years ago

eudev, elogind

snaky2• 8 years ago

Gentoo Linux is as close to BSD as possible, and have the -systemd USE flag.

ernst_klim2• 8 years ago

Because the terribly slow startup times and scripts instead of configs are such an advantage. FreeBSD people want to build a systemd-like thing as well, they just don't have enough resources atm [1] (sorry for reddit, original is in German).

[1] https://www.reddit.com/r/linux/comments/2nhkx9/freebsds_jord...

krylon3• 8 years ago

There was episode of BSDNow where they interviewed Jordan Hubbard about this: https://www.freebsdnews.com/2015/10/31/bsdnow-tv-episode-113...

Personally, I never found startup time to be much of a problem, on any OS. But having an init system that automatically restarts crashed services and is able to properly shutdown services including any child process they might have spawned sounds nice.

One thing I do like a lot about the BSD init system is that the interface is a text file with a simple format. I hope when they replace good old rc, they keep the interface as simple as that.

SpaceInvader4• 8 years ago

I have the longest beard (not yet white) in my company (1000+ plus employess), I am primarily the AIX guy (800+ virtual machines) and I do use FreeBSD since 2002 iirc. I also do plenty Linux but AIX and FreeBSD, despite being totally different, seems to be the rock solid option. Given AIX runs only on freakishly expensive power platform FreeBSD can be a good alternative. ZFS is great too.

pjmlp4• 8 years ago

I don't do Aix since 2003, does it still follow Windows model in how .so are created with export definition files?

SpaceInvader4• 8 years ago

What you mean by that? I stay away from Windows ;-)

posix_me_less4• 8 years ago

ZFS is great too.

Could you give some of your personal reasons for saying so? So much online sources just say how great it is without any specific detail. It's not like XFS on Linux has some obvious problems - it is rock solid too.

Let's say I want to run several different operating systems simultaneously from a single big ass SSD. With Linux, XFS and Xen, I can try to set up LVM and coerce Xen to use those logical volumes for VMs, but it is pain to get there with the current state of XFS and Xen documentation. Is this possible/easier with FreeBSD and ZFS?

reacharavindh• 8 years ago

Just sharing my personal experience.

Context: I use XFS as the default file system on all HPC compute nodes, and few other servers that run CentOS7. I built a FreeBSD+ZFS backup server recently to mirror our primary NAS.

The experience with ZFS was so smooth and logical. All I needed was the ZFS section from the FreeBSD handbook, and I actually understood what was going on underneath to a reasonable extent. No magic - "run this command" without explanation. Once spools are set, ZFS datasets provide the equivalent of logical volumes you were asking for. You get a lot of tunables at that level, and they come with sensible defaults. The whole process is simple, heck you can even make a file act as a zpool for learning or testing purposes.

Once done, I was testing performance rudimentarily and saw that it was almost saturating the network pipes beating my expectations without any time spent on custom perf tuning.

And I get snapshots and online compression, SW RAID out of the box.

I may be able to achieve most of it with XFS, some specific hardware and lot of hours of tuning, but it will never match the experience with ZFS.

ZFS blog post: https://aravindh.net/post/zfs_fileserver/

Performance: https://aravindh.net/post/zfs_performance/

bsder• 8 years ago

Could you give some of your personal reasons for saying so?

For one example, ZFS checksums spot all manner of erratic, intermittent idiocies--memory corruption, data bus corruption, disk controller corruption, etc.

No other filesystem even tries.

sys_64738• 8 years ago

AIX sort of died in the 2000s. Back in the late 90s the only reason to have AIX systems were as gateways to the AS400 systems. You didn't generally use AIX systems for anything else back then.

pjmlp4• 8 years ago

Zero reasons why they would make a better 3D, UI/UX stack though.

pierreprinetti• 8 years ago

I'd love to switch to BSD, but

Docker on FreeBSD is experimental.

https://wiki.freebsd.org/Docker

I don't want to run yet another operating system just to have to run VMs on top of it...

deathtrader666• 8 years ago

Why not use FreeBSD jails?

bauerd4• 8 years ago

* Because my company runs OCI images (aka uses Docker), not jails.

* Because you can't orchestrate jails with Kubernetes, and that's what my company uses.

Docker won over the industry, jails and friends did not.

INTPenis2• 8 years ago

I came into IT on FreeBSD back in 2004. Was a real fanboy and couldn't stop ranting about it. My first IT boss at the time got me into FreeBSD and PostgreSQL without actually knowing why. I had no real experience to back my opinion.

FreeBSD was my primary OS, both at work and at home.

To fast forward a bit I handed over my last BSD environment around 2012, last Solaris environment around the same time. After that I only had Linux responsibilities. Around 2014 I was running Debian as my primary OS and sent on a RHEL course in Stockholm.

The course leader sold RedHat to me so well that the very first night at the hotel I reinstalled my laptop from Debian to Fedora. Haven't looked back yet.

Been professionally building systems on Linux since before that. So I'd like to say my perspective spans over both sides of this argument. I've tried to summarize why I'd rather use Linux and why you might consider using BSD.

The only reason I can justify to use BSD is that the open source world needs competition. It would be dangerous if we only had one OS.

But all my professional sense of getting things done and keeping them stable tells me not to use it. Correction, I use OpenBSD on my router at home but that's it. OpenBSD has a reputation of being very secure (perhaps earned in part by having a relatively small user base) and I rarely need to login to make changes. Because the user space tools are horrible compared to Linux.

And I believe most of that stems from the community being much larger and therefore the software better tested and more mature.

I'm talking mainly about operational security and stability. I believe Linux does this much better than any BSD.

Less critical bugs, less issues with package management, better maintained packages and more binary packages available that don't require source building. Pretty much all things very reliant on a strong community.

More programmers available to fine tune user space tools and make them hum. A lot more docs available from hobbyists and officials.

Some of my friends today are hardcore BSD geeks and they often have to deal with system breaking bugs, kernel panics even. I can't remember the last time I saw a kernel panic in Linux. Almost every day I can see at least one issue in our chats dealing with package management or kernel and system bugs. Bugs in jails for example, or network management related to jails. They're often following the latest releases to remedy these but that's even worse than using Fedora on servers imo. Bleeding edge software just to avoid the bugs and be able to use the latest so called virtualization technology available.

SpaceInvader4• 8 years ago

Are you sure about the security thing? Less critical bugs? Do you have any data to back that up?

INTPenis2• 8 years ago

No I'm writing completely from my own experience running stable releases of Linux distros such as Debian and CentOS.

Very rarely I've had to install Fedora on some intranet to get a specific feature but I can count the times on one hand.

petre3• 8 years ago

I've never had a kernel panic on FreeBSD, had some on Linux. FreeBSD is rock solid.

snaky2• 8 years ago

Less critical bugs

Because the things like that not even considered bugs, not to say critical?

https://lwn.net/Articles/752063/

tmbsundar• 8 years ago

I love BSD.

But, last time I checked NVIDIA didn't have the right CUDA drivers for BSD. Is there any improvement on the status quo yet?

wartakode• 8 years ago

Recently i'm switchback to linux from openindiana, can't use eclipse (old) JEE on illumos, as update site are not working well.

- https://wiki.openindiana.org/oi/Eclipse

type0• 8 years ago

About FreeBSD, I've heard that its' software is lagging behind many fast updating Linux distros but is extremely reliable, how does Debian Testing compare to something like TrueOS i.e FreeBSD-Current stuff?

P.S when I say lagging behind I don't mean the server things but rather on the developer side.

hs86• 8 years ago

The base OS and the installed 3rd party software (ports/packages) are on different release models.

The base is either stable/LTS (~5 years support) or rolling if you track -CURRENT.

For the 3rd party software it is either rolling or stable in short lived quarterly branches.

Most server side software is usually pretty up to date but some desktop-related tools can be outdated for a while. You might want to take a look at [0] and especially at the commit history of some fairly popular desktops like Gnome3 [1] or KDE5 [2]. They got updated recently but it took quite a while.

[0] https://www.freshports.org/

[1] https://www.freshports.org/x11/gnome3/

[2] https://www.freshports.org/x11/kde5/

BSDs or Illumos may be good alternatives to Linux (2017)

Related Stories