Rob Pike comments on Ryan Dahl's rant

https://plus.google.com/115094562986465477143/posts/aEJE2vff4jS

by arturadib • 15 years ago

219 113 15 years ago

plus.google.com

metabrew• 15 years ago

Wait, what? Is he saying symlinks are bad? I like symlinks.

davidw• 15 years ago

He's saying "get off my lawn".

And he has earned the right to do so.

sp3324• 15 years ago

He didn't just say that, he headed up the Plan9 project to fix it. https://secure.wikimedia.org/wikipedia/en/wiki/Plan_9_from_B...

eterps• 15 years ago

He said they weren't added with proper appreciation of the Unix model and its simplifications. So he could mean that symlinks can be implemented in a more Unixy way?

saljam2• 15 years ago

Probably, c.f. bind and union mounts .

dtf• 15 years ago

"Symbolic links make the Unix file system non-hierarchical, resulting in multiple valid path names for a given file."

http://doc.cat-v.org/plan_9/4th_edition/papers/lexnames

reinhardt• 15 years ago

Doesn't this specific criticism hold for hard links too?

uriel7• 15 years ago

There are other problems with symlinks too: http://harmful.cat-v.org/software/symlinks

Every command that operates on files needs an extra flag to tell it whatever it should follow symlinks, or operate on the target, or on the symlink itself, etc.

barrkel• 15 years ago

That's rather the point. When presenting a view of the file system under some particular directory, it's nice to not have to hard-link every file into place, and instead include wholesale directory trees. That's my most common use case for symlinks.

jgrahamc• 15 years ago

I can understand why he wouldn't like symlinks. They weren't introduced until 4.2BSD and they seem like a bit of a hack. Compare them to hard links. If you move the destination file the hard link remains, but the symlink is broken and there's no simple way to figure out where the destination file was mved to. Also hard links include reference counting so you know that someone is pointing to your file. Not so with symlinks. Also, hard links mirror the permissions of symlinks.

Note that the history shows up in the ln command which originally only created hard links, and then got the -s parameter when symlinks were introduced.

praptak• 15 years ago

Also, symlinks create a concurrency hell of their own: http://en.wikipedia.org/wiki/Symlink_race

Roboprog• 15 years ago

Could be worse. Could be Windows desktop shortcuts. Fuzzy logic of target resolution? Now that's messy.

Symlinks may be ugly and/or dangerous relative to the original file system concept, but they solve problems. They are not "all good", but life would be worse without them. E.g. - application by application implementation of location aliases a la IIS virual directories (or whatever they called it) for web applications.

billswift• 15 years ago

I tried using hard links back in the late nineties when I was new to linux. Instead of creating a second link to the file it copied the file to the new directory. I discovered the problem when I modified one copy and found the second copy, which was supposed to be a hard link, wasn't changed. I don't know what went wrong, but with symlinks available, I just never bothered trying hard links again.

ADDED: Note that I avoided hard links after that, because I was worried about what would happen if I modified one file and deleted it, thinking it was a link to another, when it was actually a copy. A person could lose a lot of work if that happened. At least with a symlink, I know that something is or is not a link.

DanBC3• 15 years ago

(http://www.usenix.org/events/usenix2000/general/pikelex.html)

>Symbolic links make the Unix file system non-hierarchical, resulting in multiple valid path names for a given file. This ambiguity is a source of confusion, especially since some shells work overtime to present a consistent view from programs such as pwd, while other programs and the kernel itself do nothing about the problem.

etc etc.

filesystems are a mess, and it'd be great if something could fix them.

saljam2• 15 years ago

Symlinks are bad. They break the natural semantics of a hierarchical tree-like filesystem, and turn it into a messy half-assed graph.

What happens if you move a symlink? Will it still point to the same object? That depends on whether it's relative or absolute. What if you move the directory containing it? Do you have to recursively check for the correctness of links every time you operate on a directory? Sounds unreasonable to me... What if it links to a location which isn't mounted? Whats the definition of '..'? If you cd into a linked directory and do an 'ls ..', should that list the parent of the target or the link? How would you implement that?

The point is, symlinks are a messy kluge which probably wasn't thought out very well. You can look at what Rob Pike has to say about it himself on http://cm.bell-labs.com/sys/doc/lexnames.html

runn1ng2• 15 years ago

Uhhh.... yeah, but that's the point.

You have file at /some/path, then you put newer version of that file to /some/path, replacing the old file, and the symlink still points at the same address, and it doesn't matter what actually happened with the file itself.

It's useful enough to warrant its own abstraction.

Locke1689• 15 years ago

They break the natural semantics of a hierarchical tree-like filesystem, and turn it into a messy half-assed graph.

What you mean to say here is that it was a DAG and now it's a directed graph.

dextorious3• 15 years ago

Yes, but have you considered the ALTERNATIVES?

(that is, not only "not having anything like them").

etrain• 15 years ago

Doug McIlroy (inventor of pipes, original author of diff) was on my undergrad thesis committee. I think he would agree with Rob here.

While the design of these additional features violates the "UNIX way," it doesn't violate pragmatism. Too often in our field, perfect is the enemy of good enough. Is BSD's model and implementation of sockets perfect? Surely not. Is it good enough? From the purists perspective, maybe not. From the pragmatists perspective, absolutely. I probably wouldn't be typing this today (on my macbook pro) without the implementation of BSD sockets.

justincormack2• 15 years ago

On the other hand, signals were just a poor design. They are much nicer now on Linux with signalfd(2), which gives you a file descriptor to read them on, a much nicer interface than all the old syscalls for signals.

anatoly• 15 years ago

Well, I'm not sure signalfd(2) was feasible back when signals were added. Was it?

Today it's usual to structure your main() as some kind of select()-like loop; even if you don't need one for your main workflow, at the worst you can spawn an I/O thread and put it there. But back then, you didn't have threads. Lots of programs didn't read your input but still wanted to catch signals, e.g. to exit cleanly on kill -15. Many others read input, but without an event loop - they would just try to readchar() periodically when they had nothing else to do.

Was there an obviously better design back then?

mdellavo• 15 years ago

I absolutely agree with you.

I can't help but juxtapose this current dialog (which now includes one of the Unix forefathers) with the idea of "Worse is Better" (http://www.jwz.org/doc/worse-is-better.html). Maybe it's the Jersey in me but at the end of the day working, shipped software is all that concerns me.

mhd3• 15 years ago

The interesting bit here(I don't wanna say "problem"), is that we're actually a few steps further down this staircase of "let's ship". Unix wasn't exactly the purist ivory tower to begin with, then it merged with the partially compatible BSD, then we got some not-quite-Smalltalk GUIs on top of that, and now we're building elaborate web stacks on top (and/or instead) of that. And I don't even want to talk about mobile apps that in an almost unholy ceremony combine that again…

It's a Babylonian tower made of mud (some would say camel poo). But it's brightly painted and has a good view, and all your friends live in it.

Personally, I don't wanna move out either, but I still have to turn my head every time you see the paint come off somewhere…

adestefan• 15 years ago

BSD sockets has somehow made it's way through the entire computing world and we're never going to get rid of it. Even the tiniest of embedded systems have decides that sockets are the one-true-way of network programming.

uriel7• 15 years ago

And this is so sad.

By the way, Plan 9 doesn't have sockets, it has /net: http://man.cat-v.org/plan_9/3/ip with the dial API built on top: http://man.cat-v.org/plan_9/2/dial

This means that for example, Plan 9 applications that do networking are not tied to a specific network stack, you can mount multiple network stacks concurrently, you can have 'virtual' network stacks (for example running in user space and proxying to a remote host, or doing other neat tricks), and the apps don't need to care.

Also when Ipv6 was added to the Plan 9 stack, no application code had to be modified, because the API nicely abstracts network addresses.

yxhuvud3• 15 years ago

I'm curious: What is the alternative?

nicolaus• 15 years ago

I recall this missive from Linus Torvalds on the design of Linux:

"If you want to see a system that was more thoroughly _designed_, you should probably point not to Dennis and Ken, but to systems like L4 and Plan-9, and people like Jochen Liedtk and Rob Pike. And notice how they aren't all that popular or well known? "Design" is like a religion - too much of it makes you inflexibly and unpopular."

http://kerneltrap.org/node/11

uriel7• 15 years ago

That is a somewhat silly comment from Linus given that ken and rob designed and built Plan 9 together. (For example, ken designed UTF-8 while rob helped write the code: http://doc.cat-v.org/bell_labs/utf-8_history )

yxhuvud3• 15 years ago

Yes, like Apple.

.. oh wait.

It is rather that good design make you popular and not so good design doesn't.

sp3324• 15 years ago

Plan9 was a project that intended to be more Unix than Unix. For example, when Unix says "Everything is a file... except sockets, windows, etc." Plan9 says "no really, everything is a file." Here is a list of things that are not in Plan9 http://c2.com/cgi/wiki?WhatIsNotInPlanNine Oh, and did I mention Pike was one of the original leaders of the Plan9 project?

jrockway• 15 years ago

Ultimately, today's UNIX is not that far from this ideal. In Linux, you can make pretty much everything be an fd: signals can be delivered over a descriptor (signalfd(2)), filesystem change notifications can be delivered over a descriptor (inotify(7)), and so on. Everything ends up being something you can pass to select (or a variant), and so you don't really need to handle the different concepts in different ways: a modern Linux application reads from a bunch of fds and takes action based on what it receives.

Incidentally, this is node.js' model.

sp3324• 15 years ago

A lot of features were developed in Plan9 and then "backported" to other Unixes and Linux. /proc, UTF-8 etc. https://secure.wikimedia.org/wikipedia/en/wiki/Plan_9_from_B...

adobriyan• 15 years ago

> Unix says "Everything is a file... except sockets, windows, etc."

Unix said that, but in fact meant "everything has a unique descriptor associated (an integer)".

Directory is not a file, it's a set of files and other directories. Unix doesn't even tries to camouflage this simple observation and returns EISDIR.

kragen3• 15 years ago

Even in 6th Edition Unix, devices were files in the filesystem as well. And in 6th Edition Unix, you could open and read a directory as a file; that was how you found out what its children were. There was no EISDIR. I believe this behavior persisted until at least SunOS 4.

Adding sockets and windows outside the filesystem is one of the things Rob is specifically complaining about here, and in fact complained about at the time as well.

astrobe_• 15 years ago

The philosophy is that everything "looks like" a file, and this achieved through the standard file API (open/read/write/close). AFAIK sockets can be used with this API, as well as directories under Linux. The abstraction leaks quite a little bit sometimes, though.

naner• 15 years ago

shell command line history roll your own: fn history {grep '^term%' /mnt/wsys/text|sed -e 's/^term%//'}

oh yes that is so elegant...

sp3324• 15 years ago

It's DRY writ large. With that one command you get free text history for every program you use. You don't have to include history handling in each program individually.

brlewis• 15 years ago

For people who have a hard time seeing the image, here's the text of the comment:

Rob Pike - It's a different kind of mess, and for different reasons, but the Unix/POSIX/Linux systems of today are messier, clumsier, and more complex than the systems the original Unix was designed to replace.

It started to go wrong when the BSD signal stuff went in (I complained at the time), then symlinks, sockets, X11 windowing, and so on, none of which were added with proper appreciation of the Unix model and its simplifications.

So let the whiners whine: you're right, and they don't know what they're missing. Unfortunately, I do, and I miss it terribly.

mannylee1• 15 years ago

Click on the image. It should bring it up as a readable image.

decklin• 15 years ago

Not if you're using a screenreader.

antirez2• 15 years ago

Of all the things he mentions for sure signals are the worst thing. To do such a simple thing of delivering a tiny bit of information to a process the implementation and semantics of signals is so bad that you have to use atomic variables, deal with interrupted system calls, write reentrant functions, and so forth.

jballanc• 15 years ago

It's funny that you should bring that up... During this whole back-and-forth I can't help but think that Apple's libdispatch solves most of the issues brought up by both sides:

* Use a dispatch queue and dispatch_async a block to handle computation. Since the queues work from a shared thread pool and are managed at the kernel level, a long-running Fib calculation won't hold up other blocks from processing on any concurrent queue (it would still hold up a sequential queue...obviously).

* Use dispatch_sources to do your IO. Now you don't have to worry about clumsy callbacks, and your IO operations are perfectly happy co-existing with your long-running computations.

* Use dispatch_read/dispatch_write to persist to disk and you don't have to worry about different latency and throughput characteristics of files vs sockets

...and to the point about signals:

* Use a signal dispatch_source to do things in response to signals that you would never have thought possible. This is possible because libdispatch sets up a trampoline to catch the actual signal and then turn around and queue your dispatch_source block. It's not a complete panacea, but it's close...

...now if only Linux would implement kqueue

bdonlan2• 15 years ago

What does kqueue give that epoll doesn't?

Note also that signalfd() can do something similar to your dispatch_source.

saurik• 15 years ago

Ok, want to know something ironic? ;P

Internally, libdispatch is built on a system call named "kevent" (part of FreeBSD, first introduced in 4.1, meaning it was introduced in 2000). This system call will block while waiting for an event.

When a system call blocks, it is subject to a signal interrupting it, in which case you need to check for the EINTR error condition to restart the system call manually, unless you have SA_RESTART set on your signal handler /and/ your system call is one of a small set of "standard I/O" calls (note: kevent is not on this list[1]).

[1] http://www.mail-archive.com/freebsd-net@freebsd.org/msg01788...

Unfortunately, looking at libdispatch's source code, one will note that none of the usages of kevent() are correctly handling EINTR. I noticed this, for the record, while debugging the "dispatch_assume_zero(k_err)" that I had determined was being hit while running some (buggy) code of mine inside of libdispatch (it checks for EBADF, and that's it).

tl;dr signals are insidious and libdispatch didn't care ;P

kragen3• 15 years ago

He wasn't complaining about signals. Signals were in the original Unix. He was complaining about "the BSD signal stuff", which changed the semantics of signals incompatibly, in order that a program could possibly handle externally-sent signals without race conditions, and also added a bunch of signals for goofy stuff like SIGWINCH.

Signals are just a software interrupt mechanism, doubling as a way to kill runaway processes. All the things you are complaining about are problems with interrupts in general. Yet all computers since the 1960s still have interrupts, because they allow you to do things you can't do without them. And that's still true when you're talking about signals. Signals allow you, for example, to implement preemptive multithreading (with SIGALARM) or transparent transactional persistence (with SIGSEGV) or buffer overrun checking (ElectricFence, with SIGSEGV) in a userspace library.

However, they've clearly been expanded far beyond what they are necessary or even good for, and lamentably are still not usable for the primary use of hardware interrupts: I/O.

antirez2• 15 years ago

My major concern is that there is a huge difference between the actual usefulness of unix signals, that is mostly limited to killing processes or signaling tiny bits of information (like when you send a signal to reload a configuration), and their semantical complexity.

I must admit it is not trivial to replace this system with another equivalent without changing a lot of how Unix works. For instance signals are used in order to interrupt a program: if you don't make this trappable (like kill -9) then programs can't recover or prepare in any way before quitting. An alternative can be to just allow a signal handler to fire that will never return.

From this point of view you could even just have two signals, with the only goal of stopping processes, one in an hard way (SIGKILL) and one in a soft way (SIGINT), firing a signal handler (but the process will terminate when the handler returns). This way you don't need handling the interruption of syscalls nor writing "safe" signal handlers. As you said signals are interrupts. If the interrupt handler can never return a lot of problems disappeared.

All the other tasks currently performed with signals should use a more general and saner message bus, with select(2)able file-alike interface.

mmahemoff• 15 years ago

"(It appears you cannot link to comments in G+…so I’ll just leave this screenshot)"

Pity.

seclorum• 15 years ago

Too bad the comments aren't just exposed as a file descriptor somewhere ..

franksalim• 15 years ago

Or a URL.

hello_moto2• 15 years ago

When I was a teenager, my bubble only talks about Operating Systems: Linux, BSD, Amiga. OSNews.com was the "it" place next to SlashDot.

It was fun back then.

I think I'm going to grab my time machine and warp back.

hello_moto2• 15 years ago

This is not sarcasm by the way. I like being nostalgic and apparently I like system related stuff as opposed to web programming. But keep doing what y'all did.

michaelchisari• 15 years ago

You forgot BeOS, which lives on in Haiku. One of my favorite "alternative" operating systems of the time. I still wish I had more hours in the day so I could hack on Haiku once in a while.

http://haiku-os.org/

jberryman• 15 years ago

A G+ post of an image of someone's comment about someone's rant. Can we be done with this "topic" please?

dextorious3• 15 years ago

Yes, because the origin and presentation of the comment matters so much more than the contents of the comment itself, right?

jmderm• 15 years ago

I don't quite understand the reason for thinking BSD sockets are horrible?

tsuyoshi• 15 years ago

I would guess the problem with sockets is in the whole creation process. Instead of just calling open() on a magic pathname (like how you deal with devices in Unix), you call socket(), then bind()/listen()/accept() on the server side, or connect() on the client side. Luckily they didn't totally screw it up, and it's just a regular file descriptor after that point... except on Windows, where they did screw it up.

riobard• 15 years ago

I guess it's because sockets are not files?

mhd3• 15 years ago

Now we just need to get Niklaus Wirth complaining about ancient Unix' complexity to close the circle…

Ok, maybe Chuck Moore complaining about Oberon after that. Can we go any deeper?

(Also: Discussing this on the web is a wee bit ironic.)

achompas• 15 years ago

So this flies over my head a bit. Can someone explain why:

(a) Ryan Dahl is qualified to speak about these issues (i.e. he worked on the original Unix design or something)

(b) Rob Pike's comments are a big deal to Ryan, and

I'm one of the "new guys" that doesn't know what he's missing, and I'd love to put this into context.

chernevik• 15 years ago

Re: b

http://cm.bell-labs.com/cm/cs/upe/

hvs2• 15 years ago

(a) He is qualified to speak about these issues in so far as he is the designer of node.js and is a programmer.

(b) Rob Pike's comments are a "big deal" because Rob Pike is one of the "original neckbeards" that worked on Unix, Plan 9, and has recently been developing the Go language at Google. i.e. He helped developed the OS that Ryan is complaining about.

(c) "right or wrong" depends heavily on what you are trying to accomplish. Much like anything in the real world, there is no right or wrong answer.

enneff4• 15 years ago

'Rob Pike is one of the "original neckbeards" that worked on Unix'

Do you know that 'neckbeard' is a slur? It refers to someone who tries to grow a beard but can only grow hair on below their jawline, which is typical of adolescents.

FWIW, Rob has never had a beard. Ken, dmr, and bwk all have/had full beards, not neckbeards.

I am kind of surprised to find myself writing this post, but here I am.

nwmcsween• 15 years ago

Utilizing single purpose programs with a highly unintegrated scripting language breaks the whole idea of do one thing and do it well as there are now n things that do the same thing but with different trade offs. We have n flags, massive unmaintainable messes of bash, posix sh and other variants. Sun's shot with a 'managed' os with javaos was more than ahead of its time.

DanBC3• 15 years ago

Assuming for a moment that they're right:

What's the alternative? OS/2 had nice features; it got crushed. BeOS had some nice features; it got crushed. (Yes, I'm aware that they're probably still around in some OSS version.) Plan9 keeps being mentioned, but do regular users understand it? (Is there a "this is why we do things, and this is why that's good" written for people who haven't written their own compiler?)

Are people working on experimental new OSs to "fix problems" in existing systems, or have we gone way past the point of no return? Is there any work on new microprocessor architecture? How much stuff in my modern OS is there because of legacy 8086 stuff?

I think, but I'm not sure, that the length of time it's taken people to get (for one example) IPv6 rolled out shows that change is not likely.

Plan9 LiveCD and installer: (http://cm.bell-labs.com/plan9/download.html)

Haiku: (http://haiku-os.org/)

uriel7• 15 years ago

> Plan9 keeps being mentioned, but do regular users understand it? (Is there a "this is why we do things, and this is why that's good" written for people who haven't written their own compiler?)

Yes, you can start with the main Plan 9 paper: http://doc.cat-v.org/plan_9/4th_edition/papers/9

And follow up with The Organization of Networks in Plan 9: http://doc.cat-v.org/plan_9/4th_edition/papers/net/

And The Use of Name Spaces in Plan 9: http://doc.cat-v.org/plan_9/4th_edition/papers/names

This will give you a good overview of the basic design decisions in the system and their rationale, for further details on how Plan 9 deal with issues from toolchain design to authentication and security see the rest of the papers: http://doc.cat-v.org/plan_9/4th_edition/papers/

They are a wonderful read even if you never touch Plan 9, they are full of insights, ideas and criticisms of existing approaches, and many even include discussion on how to apply them to existing nix systems (sadly most of this has gone almost completely ignored by the nix community).

oconnore2• 15 years ago

    >> Plan9 keeps being mentioned, but do regular users
    understand it? (Is there a "this is why we do things,
    and this is why that's good" written for people who
    haven't written their own compiler?)

    > Yes, you can start with the main Plan 9 paper:
    http://doc.cat-v.org/plan_9/4th_edition/papers/9

So, he asks about regular, ipad-toting, angry-birds-playing, how-do-I-turn-on-my-printer users and you link to a technical paper containing buzzwords such as "compilers", "internet gateways", "distributed systems", "POSIX", and "remote procedure calls". Do you see the problem with this?

peterwwillis• 15 years ago

Oh look. And old operating system idealist is agreeing with a newbie that everything's not the perfect way they want it.

I don't care.

Learn to use what you have or get something else and stop fucking complaining. The End.

Rob Pike comments on Ryan Dahl's rant

Related Stories