How can the Linux kernel compile itself?

http://stackoverflow.com/questions/494372/how-can-the-linux-kernel-compile-itself/

by dchester195 • 13 years ago

34 31 13 years ago

tcoppi• 13 years ago

This is why I think more people should expose themselves to hardware design, CPU design, and compiler writing, if only at an abstract/academic level. I've worked with a lot of entry-level programmers, and even some that were not so entry-level, that wouldn't have a clue how to answer this question, which is quite depressing.

dublinben• 13 years ago

I'm not a programmer at all, and I found this answer quite obvious. Where else does new binary software come from, if not from a computer running older software?

lutusp4• 13 years ago

It shouldn't be such a mystery, because it follows a biological model -- small, simple organisms produce more complex descendants by way of natural selection, all fueled by proteins. The same process creates complex software systems out of simple predecessors, all fueled by coffee.

Someone• 13 years ago

If you think through, it is not _that_ obvious; it can't be turtles all the way down. So, where did the first binary software come from?

The answer is that somebody punched in a program from a computer's console, probably somewhere in the early 1960s, but possibly earlier. That program was used to make it easier to enter other programs, etc.

Given that copying working code is so simple, chances are that most of the currently running hardware can be traced back to a few persons entering a boot loader by hand.

That might even be one person. For example, Windows NT was initially developed for Intel's i860 (http://en.wikipedia.org/wiki/Windows_NT#Development), so chances are that it was bootstrapped from a Unix machine. Because of that, it may trace back to the same initial boot loader as, for example, Mac OS X.

chris_wot4• 13 years ago

If he thinks that's weird, then he ought to read Ken Thompson's article about trusting compilers.

http://cm.bell-labs.com/who/ken/trust.html

vowelless• 13 years ago

A somewhat related albeit primarily entertaining idea is that behind the Quine: http://en.wikipedia.org/wiki/Quine_(computing)

IgorPartola• 13 years ago

While I love the story, it's really amusing to watch how it gets re-discovered on HN every few years, blows everyone's minds, and then gets quoted all over the place, even where it is only mildly relevant. This might be our "does glass flow" meme.

chris_wot4• 13 years ago

Seemed relevant to me. The question asked about compilers and the kernel is what compiled the compiler. Ken Thompsons' talk related to hiding code by adding code to a compiler then compile the compiler, remove the code then continue to compile the compiler with the modified version that doesn't have the source code - your modification remains.

Point being: both relate to compiling compilers.

dllthomas3• 13 years ago

Obligatory DDC link: http://www.dwheeler.com/trusting-trust

lutusp4• 13 years ago

How can the linux kernel compile itself?

Short answer: it can't. Each software system relies for its own existence on a predecessor, usually one of smaller scope. Sort of like biology and evolution.

When I wrote Apple Writer in 1978, there were no high-level development tools available, just a primitive assembler built into the Apple II BIOS. By making direct hand entries, I used the BIOS assembler to write a better, faster assembler that supported alphabetic labels. Then I used that assembler to write a better one, and eventually I had a development tool suitable for writing applications.

Linux development has relied on a similar bootstrap process, on a larger scale.

FedRegister3• 13 years ago

Just out of curiosity, have you ever written a book about the process of bootstrapping an Apple II development environment and the little discoveries you made? I would read that in a heartbeat.

lutusp4• 13 years ago

Just out of curiosity, have you ever written a book about the process of bootstrapping an Apple II development environment and the little discoveries you made? I would read that in a heartbeat.

Thanks for this suggestion! I write a lot of articles, but I haven't written much about that period, possibly because other people were writing it for me. But I think it might be interesting to write about that time, how we created software before we had access to high-level languages.

In the meantime, here's an old piece I wrote in those days, even though not quite what you have in mind:

http://www.atariarchives.org/deli/cottage_computer_programmi...

mkohlmyr• 13 years ago

Link to Ken Thompson, link to Quines and that one dude who always goes 'When I wrote Apple Writer...' - all in the first few comments.. thread complete?

niuzeta• 13 years ago

i'm still waiting for some way to tangentially connect this issue with NSA backdooring.

Robin_Message• 13 years ago

I guess you found one.

username42• 13 years ago

The bootloader can compile linux before booting : http://bellard.org/tcc/tccboot.html

XorNot• 13 years ago

This does make me wonder about something though: is there a crashplan for a compiler? Can a very minimal gcc be written which would be capable of bootstrapping the entire gcc compiler?

It seems like it would be relevant if a new architecture was developed at least.

FedRegister3• 13 years ago

If a new architecture was developed you could bring up the whole stack by cross-{assembling,compiling} from an already supported target. By doing so you could test the output using a simulator (or actual hardware) and then when the assembler/compiler is stable enough you could then cross-assemble/compile the assembler/compiler and then be self-hosting on the new architecture.

PeterisP2• 13 years ago

For any practical purposes, it is best to simply use the full complex software stack on another computer to generate binary code meant for that new architecture.

If we'd really need to bootstrap a compiler from scratch, then a possibility would be to hand-code a Forth system, and then find/modify/make an assembler written in forth; then write a compiler for some small language (scheme?) in that assembler, and then from scheme you can easily get to some C compiler that can compile the full GCC.

seclorum• 13 years ago

Can a very minimal gcc be written which would be capable of bootstrapping the entire gcc compiler?

Yes. In fact, this is how gcc is built - in stages. First stage: build a compiler that can build the compiler. Recompile the compiler with the newly built compiler. Do it again. Repeat until completion. ;)

OnACoffeeBreak• 13 years ago

Another interesting thing to remember is that an embedded operating system (Android phone OS and software, for example) is likely being built on a PC set up with a cross-compile environment and not directly on the embedded hardware.

gnarbarian• 13 years ago

http://en.wikipedia.org/wiki/Self-hosting

How can the Linux kernel compile itself?

Related Stories