Whatever happened to the Hurd? – The story of the GNU OS

http://www.linuxuser.co.uk/features/whatever-happened-to-the-hurd-the-story-of-the-gnu-os

by vorbote • 14 years ago

137 87 14 years ago

linuxuser.co.uk

sneilan• 14 years ago

It sank like a turd.

muyuu• 14 years ago

Turds float.

dalke• 14 years ago

... like a coprolite?

vacipr2• 14 years ago

For those interested, Debian is planning on releasing a Hurd variant just like they did with kFreeBSD. http://www.debian.org/ports/hurd/index

0x02• 14 years ago

Is anyone using Debian/kFreeBSD for serious workloads?

loeg4• 14 years ago

No.

Avshalom• 14 years ago

There's also ArchHurd http://www.archhurd.org/

vacipr2• 14 years ago

Thank you.I knew there was something else.

nasalgoat• 14 years ago

An oft-quoted phrase that applies here is "the enemy of good is 'better'".

Clearly RMS wanted his idea of perfection, and almost 30 years on, perfection remains out of reach while "good enough" rules the world on Linux.

An important lesson to learn.

slurry2• 14 years ago

It's easy (and legitimate) to blame the failure of Hurd on poor design and management choices.

I think it's also instructive, though, to observe how no "better than Unix" project has really attained any success. Some of them have delivered more workable code than Hurd, but Plan 9, Inferno, Amoeba - none of them have really caught on.

InclinedPlane• 14 years ago

Interestingly unix derivatives are now some of the most popular operating systems for modern devices, as iOS, Mac OS, and Android are unix/linux based.

_delirium2• 14 years ago

Some of the Plan9 people comment on and off how backwards going back to Linux seems: all these warts they had fixed decades ago, back from the grave! But a bunch of them work at Google now, so have to use Linux, since Google doesn't use Plan9 systems.

RyanMcGreal• 14 years ago

Worse is better: http://www.jwz.org/doc/worse-is-better.html

mitchi• 14 years ago

This is from Voltaire. "Le mieux est l'ennemi du bien"

JamisonM• 14 years ago

Actually the mistake that set the project back the most was the opposite, from the article:

Stallman later admitted, “I take full responsibility for the technical decision to develop the GNU kernel based on Mach, a decision which seems to have been responsible for the slowness of the development. I thought using Mach would speed the work by saving us a large part of the job, but I was wrong.”

He made what he thought was a pragmatic choice that ended up being wrong, not the "perfect" kernel.

DanBC• 14 years ago

Can anyone tell me how computing would be different today if micro kernels had taken off? If everyone working on Linux had been working on Gnu Hurd?

klrr2• 14 years ago

Then everyone would have ran BSD.

(J/K)

Micro kernels are very hard to debug.

meaty• 14 years ago

Microkernels are easy to debug.

Compared to say an SOA platform with 200 service endpoints, a database with about 2000 tables, integration with 15 other providers with XML, CSV, SFTP and some wierd shitty non standard binary protocol. All written entirely by the lowest bidder without unit tests because they knocked the budget up. Oh and with a toolchain which barely works and no VCS system past an old corrupt source safe database full of compiled DLLs which there is no source for any more.

I've done both and am currently stuck doing the latter - I'd rather poke one of my eyes out and debug a microkernel if I had a choice.

gioele• 14 years ago

> Micro kernels are very hard to debug.

Microkernels are not harder to debug than monolithic kernels. I'd even say that they are easier to debug, much easier. (Personal experience in debugging both.)

The problem with microkernel-based OSes is, as Linus Torvarlds aptly put it, that they turn well understood memory-protection problems into not-so-well-studied IPC problems. (The actual quote is «They push the problem space into communication, which is actually a much bigger and fundamental problem than the small problem they are purporting to fix.»)

The microkernel is not the real problem here, the big issue is debugging faulty IPC sequences between the servers that implement the OS services. A problem that is almost non-existent in monolithic kernel.

HOWEVER, current monolithic kernels are facing growth problems now because of two aspect: we want fancy remote storage accessed as easily as local storage (do you want to mmap a file stored in a RAID setup implemented with SATA-over-ethernet disks?) and the fact that the process model is too leaky and so we need stronger containers like VMs (that are becoming as much leaky abstractions as the current processes). All these new features require communication between various components that were previously though and implemented as independent. This means that the IPC problems are now creeping into the world of monolithic kernels.

jacquesm5• 14 years ago

> Micro kernels are very hard to debug.

Absolute nonsense. Micro kernels are much easier to debug than macro-kernels.

The reasons are obvious to anybody that has spent more than an afternoon working with both, a microkernel is so small that the chances that your problem is in the kernel are minute compared to having your problem in user space. That means that all your luxury debug tools are available to debug the vast majority of your problems.

After the initial write the kernel code of a microkernel system is hardly ever touched. A good microkernel will limit itself to do the most limited subset that will still allow the system to function and will move everything else to regular processes.

slurry2• 14 years ago

Not very different, since we're basically doing microkernel-y things anyway using various kinds of virtualization.

It would probably be more elegant to just run Hurd or whatever instead of for example running multiple JVMs on top of multiple Linux instances on top of a hypervisor (yes, this happens), but them's the breaks.

dietrichepp2• 14 years ago

Well, people are still working on the problem. You can run some runtimes directly on Xen, like the JVM and Haskell.

http://labs.oracle.com/projects/dashboard.php?id=185

http://readwrite.com/2010/11/30/haskell-virtual-machine

emiliobumachar• 14 years ago

Stallman, the FSF and the free software ideals would probably be much more widely known. Many (most?) users of GNU/Linux call the whole OS "Linux" and honestly think Torvalds started the whole endeavour.

klrr2• 14 years ago

Note: RMS think Linux is "good enough" and Linux-libre is part of GNU, so Linux will be used for the in-development official GNU "distro".

AndrewDucker2• 14 years ago

Can someone explain to me if the problem with Hurd was that the basic idea was impossible, or it was the way they went about it?

i.e. was it the relentless restarting of the project that was to blame, or did they keep restarting because every way they approached it turned out to be impossible?

loeg4• 14 years ago

Microkernels were an immensely popular idea in academia (Mach, Minix, …) when Hurd was begun; I don't think the fundamental problem with them — performance is horrible — had been made obvious yet. Microkernel design is probably a good idea, but the message passing overhead kills actual microkernel implementations.

I've heard the Windows NT kernel described as being designed like a microkernel architecture (separate modules with clear APIs), but with direct function calls instead of message passing. I don't know if the Linux/BSD kernels are much different — I've poked at all three, and at a high level they look very similar.

mcartyem3• 14 years ago

The message passing overhead issues have been successfully addressed.

http://ertos.nicta.com.au/research/l4/ http://ssrg.nicta.com.au/publications/papers/Heiser_08lca.sl...

zimbatm• 14 years ago

I've read the same issues about performances and have always wondered how the QNX micro-kernel could power embedded devices (and the next BlackBerry). http://en.wikipedia.org/wiki/QNX

haberman3• 14 years ago

> I don't think the fundamental problem with them — performance is horrible — had been made obvious yet.

On the contrary, the idea that performance of microkernels is "horrible" is the current received wisdom, believed by the majority of programmers without critical examination, and based on very performance-poor early microkernel designs like Mach.

The truth is that modern microkernel designs like L4 can perform IPC over 20 times faster than Mach. Another important advance for microkernels are tagged TLBs which can alleviate the TLB misses that are usually incurred on every context switch.

Someday, hopefully not too far in the future, someone is going to write a modern, practical, and free microkernel-based OS that implements all of POSIX with performance the rivals Linux, but that offers a level of isolation and modularity that Linux could never offer. When that happens, a lot of people are going to scratch their heads and wonder why they believed the people who told them that microkernels are "fundamentally" slow.

I had hoped that HelenOS would become this (http://www.helenos.org/), but its lowest-layer ipc abstraction is async-based, which I think is a mistake. One of L4's innovations was to use sync IPC, which offers the highest possible performance and gets the kernel out of the business of queueing. You can always build async ipc on top of sync without loss of performance; this puts the queues in user-space which is where they belong (they're easier to account for this way).

I asked the HelenOS people why they decided to go this way, and this is their response (one of their points was "the L4's focus on performance is not that important these days", which I disagree with). http://www.mail-archive.com/helenos-devel@lists.modry.cz/msg...

_delirium2• 14 years ago

One thing that's interesting is that, while microkernels haven't really won, Xen-style paravirtualization, which in a way is an even more radical version of the de-monolithization idea [1], has taken over in a lot of niches, despite performance being an even bigger problem, especially without hardware assistance.

[1] This paper, for example, argues that VMMs are "microkernels done right": http://static.usenix.org/event/hotos05/final_papers/full_pap...

jacquesm5• 14 years ago

> I don't think the fundamental problem with them — performance is horrible — had been made obvious yet.

Microkernels can be just as fast as macrokernels and other than raw throughput can give latency guarantees that a macrokernel can't.

Message passing overhead can be minimized (by using the paging mechanism for passing messages between tasks on the same node).

tzs• 14 years ago

At the time Linus was ranting about micro kernels, QNX, a micro kernel, soundly kicked the Linux kernel's butt in pretty much every way.

0x02• 14 years ago

Mac OS X' kernel is Xnu, http://en.wikipedia.org/wiki/Xnu , which is based on the Mach microkernel but with pieces and bits from FreeBSD bolted on, for a hybrid approach.

rogerbinns3• 14 years ago

They emphasised architectural elegance over short term practicality - ie do it right rather than do it quickly. The problem with operating systems is that you need to have a minimum set of working functionality in order for it to be somewhat useful to people not developing the operating system itself. You need a way of storing data, you need a reasonable number of commands to run, you need editors etc. On the hardware side there was immense turnover in the late eighties and nineties. Processor architectures soared and waned, it wasn't uncommon for cpu speeds to double, memory become increasingly cheaper, peripherals changed (mice, cdroms, floppies changed size, Zip drives, tape drives), popular hardware buses changed (ISA 8/16 bit, ADB, VLB, PCI, PCMCIA, USB, serial, parallel). Just keeping up without changing your operating system core functionality was a lot of work!

The GNU project tended to work very cathedral like, while Linux was very bazaar like. The latter meant people could support and update the kernel for their own devices ("crowdsourcing" in today's terminology). http://www.catb.org/esr/writings/homesteading/cathedral-baza...

To give you an idea of how many shortcuts Linux took, the way the original kernel worked was that it used a single large address space. Each process was slotted into 64MB of that address space. That left you with a maximum of 63 processes. A context switch just involved changing protections to enable the appropriate slot. This is far more lightweight than how Linux now and other operating systems do it with no such process limits and a far more heavyweight full address space switch. Back then it made Linux really fast at process switching. This is an example of "worse is better": http://en.wikipedia.org/wiki/Worse_is_better

What the detractors didn't realise is that the initial simplistic Linux implementation could evolve away the constraints, and were instead betting on up front "intelligent design".

zby• 14 years ago

Isn't that ironical when you consider that user space hardware controllers mean much less coordination needed - i.e. be much more bazaar style?

jacquesm5• 14 years ago

Not only was it perfectly possible, microkernels have been sold commercially for decades and are extremely popular in hard real time environments where you want to be able to pre-empt a driver as though it is just another user process (which they are!).

qxcv• 14 years ago

This XKCD[0] explains the issue with Hurd perfectly, more-or-less echoing the ideas rogerbinns expressed. Note the mouseover text, which makes the observation that the Hurd loop is the loop in which you "code right" and the Android loop is the loop in which you "code fast".

[0] https://xkcd.com/844/

frozenport2• 14 years ago

Linux had a working version when the FSF had a longterm vision. There is nothing impossible in their design and more complex systems such as Windows or Inferno/Plan9 exist. In the case of Windows imagine how many hours were spent in its design. When Linux came out it worked and everybody was shocked, Hurd was never able to do this.

frozenport2• 14 years ago

I think there is another story: Not that many people where working on GNU.

zanny3• 14 years ago

The story mentions this. The industry had a working Linux and jumped on board. It is like how the industry jumped on C++, or x64, javascript, or unicode. The small imperfections in form of these tools didn't, at the time of their inception, justify the more massive immediate workload of trying to completely replace them with no backwards compatibility for something without the blemishes. Competitors weren't mature enough to step in, so the imperfect progressions of tried methods took the reigns rather than bolder, newer ideas, that meant shifting some of the inertia of the industry.

I think that might be one of the lesser appreciable legacies of 80s - 00s in software. We have tools that are now really showing their flaws but we built the empire on cracked bricks and the unstable foundations force us to throw more man hours and effort into keeping the whole thing standing than if we just started with a fresh foundation when the better alternatives presented themselves, even if they would have taken some more work.

I still really wonder where we would be if we had a C++ with a clean grammar and Python level readability. Where we didn't try to layer interpreters and JITs over convoluted C ABI compatibility.

chipsy• 14 years ago

I recommend the "STEPS Toward Expressive Computing Systems" reports for more current-day thoughts on OS redesign (available here http://www.vpri.org/html/writings.php )

Basically, by designing exactly the necessary language for each layer, they've reduced the code requirements to reach useful applications by multiple orders of magnitude.

I honestly wouldn't be surprised if the results of this project eventually creep into industry.

drivebyacct2• 14 years ago

So if things like Arch Hurd exist... Are we just missing the user interest in Hurd or are there usability problems with it or?

mcartyem3• 14 years ago

It's not enough to build something that's better. You also need to sneak it in like a Trojan horse into a user's fortress. You can't easily launch a frontal attack on Linux.

One of the most valuable opportunities with mobile is that current technologies can become irrelevant. A big success for L4 is that it made it into mobile phones.

wtracy• 14 years ago

AFAIK a lot of it is due to issues as simple as Linux having more device drivers than Hurd.

derleth4• 14 years ago

> Linus Torvalds had begun his project to write a UNIX-like kernel for the IBM 386.

IBM 386? What in the world is an IBM 386?

kcbanner• 14 years ago

http://en.wikipedia.org/wiki/IBM_386SLC

derleth4• 14 years ago

> http://en.wikipedia.org/wiki/IBM_386SLC

Very interesting. Thank you.

chimeracoder• 14 years ago

As someone who's in his 20s and first got interested in programming as a kid by trying[1] to install Linux on his 386... I don't know how to react to this.

On the other hand, it's ridiculous to think of how far things have come since then... people who complain about Linux usability and driver issues in 2012 really don't know what they missed!

[1] and failing!

btgeekboy• 14 years ago

I remember having to get out the specifications for my monitor during the Linux setup process in the late 90s. I don't miss that.

derleth4• 14 years ago

I know what an 80386 is. I just didn't know IBM licensed the design from Intel and made their own IBM-branded parts.

jacquesm5• 14 years ago

The first IBM computer to use a 386 was iirc the ps/2 model 80. There were also some clones out in short order, most notably the Olivetti and the Apricot.

hapless• 14 years ago

An IBM-compatible with a 386, as opposed to the legions of 286 systems already able to run SCO/Coherent/MINIX

lucian3032• 14 years ago

“My first choice was to take the BSD 4.4-Lite release and make a kernel. I knew the code, I knew how to do it. It is now perfectly obvious to me that this would have succeeded splendidly and the world would be a very different place today."

So true. It's unfortunate AT&T/Unix System Laboratories kept the BSD kernel code locked up in a lawsuit. Would have loved the article to have had a deeper insight into that as Stallman had already chosen to abandon Hurd by the time Linux came around.

It wasn't just that Linux was available, it was that BSD wasn't. It's too bad that the last few years have seen a decline in FreeBSD / other BSD OS's especially as it is an amazing operating system still light years ahead of GNU/Linux is many areas. Not to mention that it's one unified OS rather than hundreds of GNU/Linux distros.

One can only imagine what would have happened had the BSD code not been tied up in lawsuits. I bet they would have gone with the mature BSD kernel, leading to a better OS, and Linux would probably be a footnote in history if that.

Interesting how inferior technology wins a lot more than it loses. That said, I still love GNU/Linux. :)

lvillani• 14 years ago

> Not to mention that it's one unified OS rather than hundreds of GNU/Linux distros.

I admittedly haven't used any BSD enough to make a well informed opinion but I was under the impression that BSDs are fragmented at the OS level (i.e.: different kernels), while Linux is fragmented at the distribution level (i.e.: default collection of software, file-system layout, etc).

I imagine that, in addition to there being different kernel flavors there are also distribution level differences (e.g.: there are subtle differences between FreeBSD's rc.conf and NetBSD's) so I'm not sure which approach is better or worse, but I tend to lean on the "one kernel, several distributions" camp.

lucian3032• 14 years ago

There is no OS level fragmentation. Sorry that makes no sense in the BSD world. FreeBSD is an OS. OpenBSD is an OS. etc. That's not fragmentation. They are different OS's. It's like saying there's fragmentation between Windows and OS X.

GNU/Linux is one OS with hundreds of distros. That's fragmentation.

jbuzbee• 14 years ago

> One can only imagine what would have happened had the BSD code not been tied up in >lawsuits. I bet they would have gone with the mature BSD kernel, leading to a better OS, and >Linux would probably be a footnote in history if that.

Likely. For myself, in 1993 I recall being totally underwhelmed with Windows 3.1 when I got my first 386 SX PC. I went looking for something better and came across some BSD derivatives and Linux. I knew that there were some vague legal issues with BSD, so I chose to download the 30 or so SLS floppies that made up a Linux "distribution" and I haven't looked back since.

gus_massa• 14 years ago

I think that this article underestimate the ability of Linus Torvalds to manage the Linux developers community. Leadership, dispute resolution, technical choices, compatibility or new features, and teven flamewars. It's very difficult to be a BDFL.

The idea is that if Linux were not available, the open source community would have been working in Hurd and not in Linux. But without Linus perhaps the community could not exist (new member leave after a few quarrels) or all the effort is lost in complete rewrites.

Whatever happened to the Hurd? – The story of the GNU OS

Related Stories