systemd's syslogd replacement supports JSON output

http://www.freedesktop.org/wiki/Software/systemd/json/

by moonboots • 13 years ago

31 52 13 years ago

dsr_• 13 years ago

This is of course exactly the killer feature that would convert me into a true systemd believer. The only thing better would be to have an ASN.1 representation of the logs as the native format, for the sake of portability. After all, if you don't have tools to analyze your logs for you, you might as well not even have logs.

duck_typer• 13 years ago

Is this a good time to define Poettering's Law - any simple and useful Unix service will eventually get rewritten into a monstrous uber service that requires a supercomputer just to run, what should be, a simple service.

kzrdude• 13 years ago

You coin the law, and you can put your name on it. Let's see if it holds up.

FooBarWidget9• 13 years ago

Which is easy for you to state, were it not for the fact that the "simple service" is broken in various ways.

Let's start with the implementation-specific brokenness. I configured a log host with rsyslog. Rsyslog's configuration format, as well as documentation, is completely abysmal. And once in a while, rsyslog would use 100% CPU for no reason. A quick strace reveals that it keeps reading or writing to a file descriptor, but getting EAGAIN as error. Looks like a bug to me. And oh yeah, there's a race condition in rsyslog with regard to privilege lowering, so that sometimes it would great directories owned as root, and then it would lower its privilege, and then it would be unable to write to its own created directory. Just great.

The above are all fixable. But what bothers me more is that syslog is entirely unauthenticated. Any process can write to syslog, claiming that it is any process. A malicious user can write a syslog entry saying that "init[1]: the system is unstable. please reboot now!". As an administrator, you will not be able to tell whether the message actually came from init (or PID 1) or not. A quick look at systemd's journal mechanism reveals systemd at least has thought about these problems: http://www.freedesktop.org/software/systemd/man/systemd.jour.... It would log the process's PID, UID, GID and more, and the sender is unable to fake that.

spc4764• 13 years ago

systemd can log the PID, UID and GID of a message because its using a Linux-specific feature of datagram based local sockets; such a feature cannot be supported on other Unix systems. But there are other reasons to use syslogd/rsyslogd/syslog-ng besides logging to local files, and that's to forward the logs to a centralized logging service. Of course, there are issues with that, as the default syslog protocol also allows any process, anywhere, to claim it is any other process (and if root, even fake its own source address) but it's being addressed (I know rsyslogd is, I'm not sure about other syslogd replacements).

My issue with systemd is that forwarding logs to a central server is an afterthought, if even that. And the problem with that is that it won't be as secure or work as well if it's part of the design. And another issue with the "afterthought support for remote logging" is that I still need to run syslogd/rsyslogd/syslog-ng because I have a ton of other network equipment (routers) that can forward their logs via the syslog protocol (which I have used on networks I've managed---by doing that, I've been able to receive notifications of OSPF routing changes, for instance). And if I have syslogd/rsyslogd/syslog-ng already running, what are the benefits of systemd logging?

asveikau3• 13 years ago

Reading this I thought: how does one authenticate a pid or uid in a daemon? I am assuming Unix domain sockets, which don't provide you with the caller's pid or uid.

Doing some googling I found there are indeed nonportable interfaces for this. First I found getpeerucred which is a Sun thing. Then I found Linux has a SO_PEERCRED socket option. [Edit: and there seems to exist getpeereid and LOCAL_CRED on some BSDs]

Still seems like a weird or otherwise shaky thing to me. It feels like it breaks the abstraction of a socket.

mordae• 13 years ago

No, what happened was init scripts. Something completely alien to the original init tab concept. It was an extremely ugly hack that worked only because of extensive documentation and effort by much larger amount of people than systemd has now. Both systemd and journal are an actual solution. Try journalctl on your system, it actually makes sense! The silly thing even has bash completion of filtering criteria! And they are both lightning fast.

zanny4• 13 years ago

I do want to mention binary logs piss me off, but I just set up journald to output to syslog and everything is fine. I just ignore journalctl. I personally find it to be quite slow, but I can just set it to not log its binary blobs.

fosap2• 13 years ago

something that was not necessary. Init scripts can be nice and clean [1]. And yes, I'm all for a init replacement. Something new, nice and clean. But I take the shell script hackery over a dbus dependency. But that's just my personal opinion.

I have neither used it nor looked at the code, but [2] seems to be what I want. It's by the libdietc author, so i expect the code to be small, elegant and not maintained very well.

[1] http://www.freebsd.org/doc/en/articles/rc-scripting/rcng-dum...

[2] http://www.fefe.de/minit/

zanny4• 13 years ago

I use systemd and find it to be significantly faster boot-wise than Upstart and the sysv scripts. One doesn't do socket initialization and the other is a bunch of slow ass shell scripts. It doesn't matter if systemd is beefy if the actual task of starting services is lightning fast.

Just for comparison, last year I was triple booting Debian Testing, Ubuntu 12.04, and Arch. I use KDE on all 3 (I do active development on KDE in the Arch one) and logged my boot times over 6 months. I misplaced those records but the effective boot times were around 15 seconds for Debian, 25 for Ubuntu, and 5 for Arch. Mainly because tasks in Ubuntu like NetworkManager or tmpfs mounting were taking 5 - 10 seconds and nothing else was running.

gnosis• 13 years ago

There some competing trends in the *nix world today.

On the one hand, there are some huge, monolithic packages like Xorg that have split in to many smaller, more independent packages.

On the other hand, we have something like systemd absorbing ever more functionality, becoming more monolithic with feature bloat.

I am personally more in favor of having many small, simple tools that each do one function well and can be easily integrated together, and understood independently of each other, rather than going the monolithic route.

To me, systemd is one gigantic step backwards from this vision, even if it has some (or even many) good ideas.

FooBarWidget9• 13 years ago

A quick look into systemd reveals that it's comprised of multiple components. For example the journal service is a separate daemon. What part of this does not align with your vision?

EDIT: looks like the "monolithic" myth has already been busted: http://0pointer.de/blog/projects/the-biggest-myths.html

I would say the distinction between systemd and older systems is that systemd follows the "better is better" mantra of design, while older systems follow the "worse is better" mantra of design. Older systems often feel like quick hacks. For example SysV init is a collection of init scripts. Yeah it is "simple" in that the init service itself is simple, but writing the scripts themselves is anything but simple, and they are brittle as well. How often have you seen init scripts failing during startup or boot? And why must you write daemon management code over and over again your init scripts? All this does is pushing complexity out of the init service and into the scripts. Systemd is taking complexity upon itself so that the scripts can be simple.

Tobu4• 13 years ago

It's monolithic because you can't pick and choose the components.

You have to use all of systemd, or none of it. You can't replace bits of it, or use just parts of it.

asb• 13 years ago

I agre with your final paragraph and I rather like the consolidation that systemd offers. I'm certainly not going to miss fiddling with consolekit and I'm looking forwards to managing my user session via systemd. It reduces the number of things I need to learn to have a decent level of control over my system.

chipsy• 13 years ago

Sounds like Sutherland's Wheel of Reincarnation in action. All you have to do to predict the next generation of a technology is run further along the wheel.

wereHamster• 13 years ago

On the one hand, there are some huge, monolithic packages like Xorg that have split in to many smaller, more independent packages.

You should note that this is generally looked at as a bad move. Creating two separate packages for each protocol (one headers, one implementation), separate packages for each individual app etc. lead to a lot of unnecessary overhead for everybody (developers, release managers, package maintainers etc). Over the last couple years people have suggested to merge some of the packages back into the main xserver repository.

veeti• 13 years ago

Before this thread devolves into the same arguments all over again, maybe everyone could read the following:

http://0pointer.de/blog/projects/the-biggest-myths.html

senko• 13 years ago

Lennart, please add the f*ng mail reader in there already and be finished with it.

weitzj2• 13 years ago

But what if it's used for evil? :0

tzs• 13 years ago

It doesn't look like they are using Douglas Crockford's JSON implementation, so using it for evil is probably OK.

weitzj2• 13 years ago

Thanks for the upvote. I did not know my karma would get busted for my comment.

oblio• 13 years ago

After hearing both sides of the argument over and over again (for systemd and against systemd), my only regret is that Ubuntu is not replacing upstart with systemd, so that Loonix will have a uniform boot system.

To all the systemd naysayers, please read the docs before commenting, 90% of the comments against systemd look like they've been written by someone who:

a) hasn't used systemd

b) hasn't read anything relevant about systemd

Basically most comments come straight out of the blog post about systemd myths. That's really, really sad.

static_typed• 13 years ago

The way this is going, Linux will end up as a single process operating system, that will do everything at web scale. All outputs in glorious binary format.

Funny, the crowd got distracted over Gnone3, Unity or Mir, or the latest worry of the day, yet never saw the big threat in their camp.

Linux isn't done until the text processor won't run.

teho4• 13 years ago

systemd is hardly monolithic as it is build of over 80 binaries. A normal system with systemd runs at least four processes: systemd (PID1), systemd-udev, systemd-journald and systemd-logind. There's various smaller services that are run on startup and other occasions. Then there are many configration utilities like timedatectl (systemd-timedated), hostnamectl (systemd-hostnamed) and localectl (systemd-localed).

systemd's syslogd replacement supports JSON output

Related Stories