The Tragedy of Systemd [video]

https://www.youtube.com/watch?v=o_AIw9bGogo

43 88 7 years ago

creshal• 7 years ago

The real tragedy is that so many complain about systemd and insist it's awful, yet nobody ever bothers to actually make a better init system. It's either 1980s crap that's totally unsuited to modern environments, or systemd with all its bugs and design flaws.

Bluecobra• 7 years ago

I thought that Sun did a pretty good job with SMF in Solaris 10, it’s too bad that Oracle did everything they could to ruin Solaris.

sgt• 7 years ago

They say Larry Ellison is still burning Solaris and other Sun books over the weekends. At one point his Japanese bamboo house almost caught fire.

onli4• 7 years ago

There is runit, used in void. And OpenRC in Funtoo/Gentoo. And the systems used in alternative Operating Systems, Solaris just got mentioned.

There are a bunch of better alternatives out there.

Accacin• 7 years ago

I believe in the talk he goes into why this (in his opinion) isn’t the case.

isostatic10• 7 years ago

What modern environments?

The problem with systemd is that it's replaced systems that met peoples needs with a single system that many people consider to be worse.

Perhaps there are some niche environments that systemd is needed, and that's great -- but why break things for everyone else?

loeg6• 7 years ago

To flip this around, if you replaced systemd with the old rc init in e.g. Fedora today, you could make the exact same argument:

The problem with [old init] is that it's replaced systems that met people's needs [systemd] with a single system [old init] that many people consider to be worse. > Perhaps there are some niche environments that [old init] is needed, and that's great -- but why break things for everyone else?

So the argument is kind of meaningless without a tie-breaker of some kind; perhaps popularity vote.

I think you're under the impression that your views of systemd's relative suitability are held by the majority of users, but I believe that is incorrect.

keypusher• 7 years ago

No, the tragedy is that you didn't watch the video, because the author isn't complaining or insisting that it's awful. Instead, he (a well-known FreeBSD developer) goes through the history of init systems and offers some justifications for why systemd is the way that it is, and what challenges might need to be overcome to make a "better" init.

Gby56• 7 years ago

Great talk imo, my TLDR would be : > systemd is not the best it could be, but it's there. > distros patting themselves on the back because they don't use it means they haven't listened to actual needs and stayed behind. > death threats over software that nobody else tried doing are ridiculous

episodeiv• 7 years ago

death threats are ridiculous

Fixed that for you...

mondoshawan• 7 years ago

Don't forget the gaslighting with "software has bugs" and "you just dont like change", while glossing over legit complaints about the outright narrow minded design of the system that is actively harmful in some cases (systemd-timesyncd, I'm looking at you).

Personally, the video made me rather angry because all he does is set up straw men and knock them down using the same terrible pro-systemd arguments that get trotted out every time someone has a legit problem with its design.Some of the distros that use systems like runit and openrc are using those sustems because systemd doesn't fit their technicals needs, and likely won't ever because of designs that are made without community input.

Oh, sure, he mentioned the Debian developer resignations, but outright failed to point out the reason why they resigned: systemd was being railroaded into the system as a political move rather than being judged on its technical merits.

theamk• 7 years ago

Oh, sure, he mentioned the Debian developer resignations, but outright failed to point out the reason why they resigned: systemd was being railroaded into the system as a political move rather than being judged on its technical merits.

No. Ian resigned because he tried to ignore systemd's technical merits and tried to used political moves (GR process) to to kill it. As a result, most of Debian's community told him to top messing around with politics. Ian's resignation letter [1] explicitly says that:

I should step aside to try to reduce the extent to which conversations about the project's governance are personalised. [...]
The majority of the project have voted to say that it was wrong of me to bring this GR at this time.

[1] https://lwn.net/Articles/621895/

Please stop saying systemd was being railroaded. I have no idea what was going on in Redhat (I do not follow their mailing lists), but at least in Debian, it got on its own technical merits, and Poettering had no special influence there.

(That is not to say it is without problems; there are plenty of them. Still, the lesser of evils...)

timw4mail• 7 years ago

The tragedy of having to watch a youtube video instead of reading a transcript...

loeg6• 7 years ago

https://news.ycombinator.com/item?id=19023232

LWN does a reasonably faithful summary.

ilovecaching• 7 years ago

What problems do people have with systemd? I actually think it’s pretty great. They also made some tools like busctl that are really nice for Linux developers.

isostatic10• 7 years ago

Something that annoyed me 10 minutes ago

Old fashioned way, when I restart apache having made a typo:

  Output of config test was:
  AH00526: Syntax error on line 2 of /etc/apache2/sites-enabled/mysite.conf:
  Invalid command 'blah', perhaps misspelled or defined by a module not included in the server configuration
  Action 'configtest' failed.

Now

   [....] Restarting apache2 (via systemctl): apache2.serviceJob for apache2.service failed because the control process exited with error code.
   See "systemctl status apache2.service" and "journalctl -xe" for details.
    failed!

The former is far more informative

gh02t2• 7 years ago

I mean I agree that it should just show you the output of "systemctl status" in the error message, but your comparison isn't entirely fair. The commands systemd error message give will show you the same thing as the generic sysv output.

seba_dos1• 7 years ago

busctl looks simply like mdbus made 10 years later

npongratz• 7 years ago

A litany of arguments against systemd are listed here: http://without-systemd.org/wiki/index.php/Arguments_against_...

nickik6• 7 years ago

Or other wise called a list of dependable design choice that the author doesn't like and a list of incompatibility with the tools the author used before.

evol2623• 7 years ago

The actual speaker is great, but the vast majority of people who have a problem with systemd get there philosophically.

In a "I never actually had to write any code which had significant interaction with existing solutions (openrc, upstart, sysv, bsd, whatever), but I know they exist, so I'll talk about them like they solve the same problems as systemd" way. Which they do not. Want socket activation, double forking, namespaced children, sane scripts/unit files that can build a good dependency tree? You have one option.

Additionally, people see other tools like systemd-network and say "but systemd is doing everything!" Which it isn't. Almost no distros package those as part of the default.

Even if they did, many of the anti-systemd crowd would rather live with a different, slower system (NetworkManager, for example) which is vastly most complicated than what they need. Ditto for systemd-resolved and local caching dnsmasq.

Systemd is not trying to replace NM or BIND or anything. It's basically saying "it's 2019. If you want to be able to work with a system without knowing the ins and outs of 30 different configuration files, and speed up boot time/auditability of your system significantly, you can use these services"

Some things are bad for the community in general. The dependence on dbus (and kdbus), systemd-logind, and other tools aren't very friendly to BSDs to anti-systemd distros. Frankly, from my POV, this isn't systemd's problem. The anti-distros are free to do what they want, knowing the limitations. The BSD guys have written compatibility layers, which you can do, because they're open standards.

Broadly, I'd say that the anti-systemd crowd holds to an idea of "UNIX" purity, as if HP-UX, Tru64, AIX, Solaris, IRIX, and others were uniform. The reality is that they were more different than they were alike, in terms of administrative tools, filesystem layout (is this /sbin/init.d, /etc/rc.d, /sbin/rc.d, etc?) and even software portability. A large reason that GNU autoconf exists is to detect all of the quirks on all of these platforms.

Additionally, anti-systemd people either never used X11R6/OSS/lilo and other old tools, or don't remember how terrible they were. Any large change in the community (X.org fork, ALSA->PulseAudio, grub) draws ire. From the same people who are trying to use syslinux and efibootmgr instead of just using grub2.

Some people would rather debate philosophy than get actual work done, and that's great for open source, even if it's frustrating. But those people never dug into the guts of upstart to make something work, or had 250 lines of sysvinit boilerplate+supervisord saved off somewhere to speed up a process that you can do in 4 lines in a systemd unit file.

nickik6• 7 years ago

Solaris and other Unix had things like systemd already.

antirez3• 7 years ago

The tragedy of systemd is that it's the wrong solution to a real problem. Because most people against systemd will keep saying that the old init system was ok, they don't move forward trying to provide a true alternative. So systemd became a standard despite the issues, because it solves certain real problems that people have, even if not great.

kfwhp2• 7 years ago

systemd became a standard because the guy works at Red Hat so he can make GNOME require it (there is no way to use GNOME without systemd; distros that allow that such as Gentoo do so by patching GNOME, which is costly.)

I agree that a more modern init system is necessary, but the idea of systemd is meh and the implementation is subpar.

tapoxi• 7 years ago

systemd became a standard because Debian adopted it after a lengthy discussion, partially because big Debian users like Spotify expressed an interest in systemd.

https://lists.debian.org/debian-ctte/2014/01/msg00287.html

nickik6• 7 years ago

Conspiracy nonsense. It became standard because the existing solutions were crappy and nobody else bothered to do anything about it.

kevin_b_er• 7 years ago

The intentional and willful breaking of screen and tmux was to fix a GNOME bug of GNOME not closing up as it should when the user logs out, so systemd was changed to mass kill processes. The interplay between GNOME and systemd in backroom dealings is a major pain point.

taffer• 7 years ago

You make it sound like the systemd author was something the CEO of all Linux Distros.

The maintainers of Arch Linux have given an explicit list[1] of the reasons why they decided to introduce systemd:

    0) it is hotplug capable
    1) we can know the state of the system
    2) it is modular
    3) it allows dbus/udev to go back to doing the task they are meant to do
    4) we can reduce the number of explicit ordering dependencies between daemons
    5) we get a lot of security/sandboxing features for free
    6) systemd service files can be written and distributed upstream
    7) systemd is a cross-distro project
    8) logind will finally deliver on what consolekit was supposed to do
    9) systemd is fast

[1]: https://bbs.archlinux.org/viewtopic.php?pid=1149530#p1149530

sametmax3• 7 years ago

I concur. Before systemd, I used supervisor because I wanted to avoid using any init system like hell.

Since systemd, I just do a service file.

So despite all its shortcommings, I now have a standard, well documented, init system that I can use accross all the distros I use, which happens to provide all the features I need.

I'm sure all the power user complaints about systemd are legitimate, but I really don't care about those. I just want an easy way to setup custom deamon on ubuntu and centos.

Provide a better system, that solve those problems, and I'll use it. I really don't care as long as my problem is solved.

onli4• 7 years ago

So despite all its shortcommings, I now have a standard, well documented, init system that I can use accross all the distros I use, which happens to provide all the features I need.

Same couldn't be said about supervisor, the solution you already had running (well, maybe strike init)?

upofadown• 7 years ago

Because most people against systemd will keep saying that the old init system was ok, ...

There isn't and wasn't hardly anyone saying that. It is and was well known that there were better things out there than the hacked up sysV style init systems produced by the various Linux distributions.

This is a common strawman that seems to get rolled out any time any discussion involves systemd.

antirez3• 7 years ago

A few of the authors of one of the Debian based distributions dropping systemd are friends of mine and all of them see things in that way, that the good old init is fine and there is no need to replace it. In the Facebook group where they use to stay the others I don't know first-hand seem to share this POV.

nickik6• 7 years ago

And yet basically no alternative to systemd was produced by any of those people.

joepie91_• 7 years ago

Nearly every systemd-disliking person I've personally spoken to, has trotted out that exact line about sysV init being fine.

It's not just a strawman, unfortunately. I wish it were.

isostatic10• 7 years ago

But it's not just replacing init - it's replacing a dozen other services including things like ntp or dns, and often it's replacing them with inferior solutions.

antirez3• 7 years ago

Yes, that's a design problem due to what the systemd main author believes to be a good thing.

nickik6• 7 years ago

systemd can be used without most of these things. systemd as a project has other services in but those you can use or not use.

And the claim that they are 'inferior solutions' is just absolutely false in most cases. It often replaced horrible old tools that haven't been updated in 30 years. It often provides far better interfaces and tooling then the existing solutions.

kbenson5• 7 years ago

My understanding is that the reason it started replacing other services is that there's a dependency graph that ends up pulling them in, and very early in the boot process depending on configuration[1]. That, and in some cases the available solution did not support their needs[2] (which might have been solved better through upstream work, or maybe not, if the targeted project was slow or unwilling, I'm not sure).

1: https://lists.freedesktop.org/archives/systemd-devel/2014-Ma...

2: https://www.reddit.com/r/linux/comments/6kdcme/why_does_syst...

altmind2• 7 years ago

Because most people against systemd will keep saying that the old init system was ok, they don't move forward trying to provide a true alternative.

The problem with systemd it is not just an init system. Its network-state-aware, device-state-aware container launcher(nspawn), cron, logind and syslog implementation.

The scope creep of systemd is enormous, nobody asked for that when they wanted reliable daemons.

w323898• 7 years ago

More or less this. Gummiboot is nearly pointless, and ntpd and resolved have both caused problems for me. As an init system, systemd is fine, but shipping these bad components while giving them the systemd "seal of approval" is dumb.

On the other hand, I've had no issues with logind and gdm. I've been interested in trying S6 but given it's quite easy to substitute out the bad parts of systemd, so I just haven't bothered.

Probably the biggest problem with systemd is that it's hurting BSD. Then again, gnome devs are having enough trouble getting things working on Linux... also binary format journals are so dumb. How have they not changed that?

dvfjsdhgfv• 7 years ago

The tragedy of systemd is tight coupling. What makes Linux great is that I used to always have a choice - regarding what kernel I use, what editor, what shell, even what Init I use... Systemd changed all that. Distro makers gave up supporting alternatives because it would mean too much work. (I almost flipped out when I read somewhere they had to do it because of some Gnome-related component... I mean I don't care about Gnome, I don't even use X, but I have hundreds of servers to manage - now let's change everything because one guy said it's the only way...)

Nobody would make what Lennart did if we had a choice. Those who liked Systemd would use it, and the rest would use the init of their choice. As simple as that.

ardy42• 7 years ago

Previously:

LWN article:

https://news.ycombinator.com/item?id=19023232 (1/29/2019)

This talk:

https://news.ycombinator.com/item?id=19157293 (2/13/2019)

Another youtube version of it:

https://news.ycombinator.com/item?id=17778202 (8/16/2018)

syn0byte2• 7 years ago

I ask this in every systemd thread but I genuinely would love a real answer. Can anyone give me exact real-world examples of where systemd is objectively better suited than any number of other init systems?

The fact that I still have yet to see any such examples in the last half a decade seems pretty telling. Wouldn't and shouldn't there be tons of such examples given the rhetoric?

foldr4• 7 years ago

It makes it a lot easier to set up a service than System V init. Writing a service file is much easier and less error prone than writing shell scripts.

JdeBP4• 7 years ago

syn0byte explicitly said "any number of other init systems", and was clearly trying to avoid people trotting out, as you and several others have, the tired old fallacy, called out by the Uselessd Guy long since, of assuming that only systemd and van Smoorenburg init+rc exist ...

* http://uselessd.darknedgy.net/ProSystemdAntiSystemd/ (https://news.ycombinator.com/item?id=8488235)

* https://blog.darknedgy.net/technology/2015/09/05/0/

* http://jdebp.eu./FGA/run-scripts-and-service-units-side-by-s...

... and then calling the latter "System V init", when AT&T System V had actually adopted a new system with separate system and service management several years before Linux was even invented.

* https://news.ycombinator.com/item?id=18788974

zaarn2• 7 years ago

If you want to container your services better, then systemd is a good choice; it has plenty of isolation options to secure yourself against misbehaving code. By default systemd will not leave any orphans lying around in your system if the service dies unexpectedly.

You can also manage much more complex dependency situations (like two services being dependent on each other and requiring a restart of both if one crashes).

Systemd Timers are also much more robust and predictable than piling scripts on top of cron, I use them extensively for managing backups on both servers and desktops with automatic reporting for failures.

jpdb• 7 years ago

It's much easier to manage an application's life-cycle in a systemd unit file than it is in sysvinit bash script.

Unit files are much more declarative and self-contained compared to a bash script. And the bonus is that if you need/still want to use bash, you can do so in systemd.

rkangel• 7 years ago

To echo the sibling, but with some more detail:

I worked on a system that (among other things) packaged up a Linux system as a network appliance. The device ran multiple daemons. I have built similar things before in a SysV world, and systemd made the following things easy:

* Starting each daemon only when the system resources it needed (e.g. network up) were ready

* Starting the daemons in the right order relative to each other

* Putting daemon log output in syslog

* Restarting when crashed (for those where it was appropriate)

These all required only a few lines of systemd config, and no special coding in the application. For instance, logging was just printing to stdout/stderr and systemd took care of the rest.

aequitas• 7 years ago

I would like to ask the question the other way around. Now I've been using systemd for a while and gotten used to it I can't image wanting to replace systemd with a collection of scripts.

Maybe systemd is not objectively better or worse. But I'm at least glad there is a standard among Linux systems so I only have to learn the quirks of one init system.

altmind2• 7 years ago

Systemd is great if you/the company are developing your own software that need to be daemonized. Its easy to write/copy one .service file for all the distributions than to create some shell scripts/service descriptiors for a bunch of different distros.

The alternative approach is to use supervisord - it also have service descriptors and does what systemd does. The last time we tried to use it(rhed6 had no systemd) we found it lacking some options important to us.

loeg6• 7 years ago

The fact that I still have yet to see any such examples in the last half a decade seems pretty telling.

Yes, it's telling; it suggests you've had your fingers deliberately in your ears for five years.

cr0sh• 7 years ago

What I have come to believe is the "Tragedy of Systemd" - and if I am wrong on this, please let me know - is that it was a virtually unilateral decision to make the change, by one person, without any input from others.

It was like all at one shot - here's this huge and largely non-transparent init system created by seemingly one person who it felt like was shoving it down everyone's throat. My way or the highway style.

That, on top of what felt/feels like a lack of transparency about how it works (I could look at init scripts and understand them fairly easily and how they worked together - systemd seems almost a mystery to me; I just have to trust it works - so far so good I guess).

Maybe I am wrong on all of this - again, if I am (and there's a good chance I am I feel) - please let me know.

JdeBP4• 7 years ago

You are wrong. Read https://news.ycombinator.com/item?id=14735100 .

fridsun• 7 years ago

I just must let this one thing out.

34:02 binary logs are not a bad thing as long 34:06 as you have the tools to pull them apart

Really? Did he just "as long as" the biggest downside of binary log compared to text log, that is way fewer tools work on a binary log due to its special encoding?

With the same logic, flu viruses are not a bad thing as long as you are immune to them.

The Tragedy of Systemd [video]

Related Stories