Re:How can the Linux kernel compile itself?
by lutusp on Tuesday September 17, 2013 @10:38AM
(#6399543)
> How can the linux kernel compile itself?
Short answer: it can't. Each software system relies for its own existence on a predecessor, usually one of smaller scope. Sort of like biology and evolution.
When I wrote Apple Writer in 1978, there were no high-level development tools available, just a primitive assembler built into the Apple II BIOS. By making direct hand entries, I used the BIOS assembler to write a better, faster assembler that supported alphabetic labels. Then I used that assembler to write a better one, and eventually I had a development tool suitable for writing applications.
Linux development has relied on a similar bootstrap process, on a larger scale.
Re:How can the Linux kernel compile itself?
by FedRegister on Tuesday September 17, 2013 @12:08PM
(#6400101)
Just out of curiosity, have you ever written a book about the process of bootstrapping an Apple II development environment and the little discoveries you made? I would read that in a heartbeat.
Re:How can the Linux kernel compile itself?
by lutusp on Tuesday September 17, 2013 @02:16PM
(#6400877)
> Just out of curiosity, have you ever written a book about the process of bootstrapping an Apple II development environment and the little discoveries you made? I would read that in a heartbeat.
Thanks for this suggestion! I write a lot of articles, but I haven't written much about that period, possibly because other people were writing it for me. But I think it might be interesting to write about that time, how we created software before we had access to high-level languages.
In the meantime, here's an old piece I wrote in those days, even though not quite what you have in mind:
Re:How can the Linux kernel compile itself?
by FedRegister on Tuesday September 17, 2013 @03:13PM
(#6401242)
The Atari Archives piece was part of what motivated me to ask about this. Another part is the series of books Jordan Mechner has released (in both print and electronically) about the development of Karateka and Prince of Persia. I've found that I enjoy following along the process of discovery almost as much as I enjoy the finished product. There is also a practical benefit: I gain more intellectual tools for tackling obstacles by learning the process others use to tackle obstacles.
Re:How can the Linux kernel compile itself?
by IgorPartola on Tuesday September 17, 2013 @10:44AM
(#6399572)
While I love the story, it's really amusing to watch how it gets re-discovered on HN every few years, blows everyone's minds, and then gets quoted all over the place, even where it is only mildly relevant. This might be our "does glass flow" meme.
Re:How can the Linux kernel compile itself?
by chris_wot on Tuesday September 17, 2013 @05:46PM
(#6402142)
Seemed relevant to me. The question asked about compilers and the kernel is what compiled the compiler. Ken Thompsons' talk related to hiding code by adding code to a compiler then compile the compiler, remove the code then continue to compile the compiler with the modified version that doesn't have the source code - your modification remains.
Re:How can the Linux kernel compile itself?
by dllthomas on Wednesday September 18, 2013 @06:16PM
(#6408524)
Uh, it was the parent comment - do a search if you can't find it on perusal. It's possibly taken out of context but I don't see a reading of the passage in question that doesn't include that sentiment. Could you clarify?
Re:How can the Linux kernel compile itself?
by tcoppi on Tuesday September 17, 2013 @09:16AM
(#6399063)
This is why I think more people should expose themselves to hardware design, CPU design, and compiler writing, if only at an abstract/academic level. I've worked with a lot of entry-level programmers, and even some that were not so entry-level, that wouldn't have a clue how to answer this question, which is quite depressing.
Re:How can the Linux kernel compile itself?
by dublinben on Tuesday September 17, 2013 @10:28AM
(#6399459)
I'm not a programmer at all, and I found this answer quite obvious. Where else does new binary software come from, if not from a computer running older software?
Re:How can the Linux kernel compile itself?
by lutusp on Tuesday September 17, 2013 @10:42AM
(#6399563)
It shouldn't be such a mystery, because it follows a biological model -- small, simple organisms produce more complex descendants by way of natural selection, all fueled by proteins. The same process creates complex software systems out of simple predecessors, all fueled by coffee.
Re:How can the Linux kernel compile itself?
by lutusp on Tuesday September 17, 2013 @11:21AM
(#6399781)
> The chicken or egg problem shows that at some point you lose the connection to the predecessor and the process goes around in endless circles.
No, this isn't so. It's a myth about biology that's repeated by people who don't understand natural selection. An egg is a chicken's way to make an insurance salesman.
Re:How can the Linux kernel compile itself?
by jlgreco on Tuesday September 17, 2013 @11:49AM
(#6399970)
You don't have a chicken/egg problem with either biological evolution or computers. With biological evolution abiogenesis is at the bottom of the turtle stack, with computers it is likely some guy poking holes in some cards or rewiring a computer by hand.
Ultimately you don't need life to create life, and ultimately you don't need a computer program to build a computer program. There are other methods, even though we don't see those methods executed today.
Re:How can the Linux kernel compile itself?
by Someone on Tuesday September 17, 2013 @11:15AM
(#6399750)
If you think through, it is not _that_ obvious; it can't be turtles all the way down. So, where did the first binary software come from?
The answer is that somebody punched in a program from a computer's console, probably somewhere in the early 1960s, but possibly earlier. That program was used to make it easier to enter other programs, etc.
Given that copying working code is so simple, chances are that most of the currently running hardware can be traced back to a few persons entering a boot loader by hand.
That might even be one person. For example, Windows NT was initially developed for Intel's i860 (http://en.wikipedia.org/wiki/Windows_NT#Development), so chances are that it was bootstrapped from a Unix machine. Because of that, it may trace back to the same initial boot loader as, for example, Mac OS X.
Re:How can the Linux kernel compile itself?
by PeterisP on Tuesday September 17, 2013 @06:48PM
(#6402434)
Well, to be detailed, both the first kernel and the first compiler was probably entered from a stack of punchcards or punch tape made with mechanical "typewriters"; punching a program from console is rather unwieldy and generally you should do that only for the bootloader that loads the rest of the (manually compiled) code from some electromechanical device.
Re:How can the Linux kernel compile itself?
by mkohlmyr on Tuesday September 17, 2013 @10:56AM
(#6399644)
Link to Ken Thompson, link to Quines and that one dude who always goes 'When I wrote Apple Writer...' - all in the first few comments.. thread complete?
Re:How can the Linux kernel compile itself?
by XorNot on Tuesday September 17, 2013 @11:53AM
(#6399992)
This does make me wonder about something though: is there a crashplan for a compiler? Can a very minimal gcc be written which would be capable of bootstrapping the entire gcc compiler?
It seems like it would be relevant if a new architecture was developed at least.
Re:How can the Linux kernel compile itself?
by FedRegister on Tuesday September 17, 2013 @12:04PM
(#6400070)
If a new architecture was developed you could bring up the whole stack by cross-{assembling,compiling} from an already supported target. By doing so you could test the output using a simulator (or actual hardware) and then when the assembler/compiler is stable enough you could then cross-assemble/compile the assembler/compiler and then be self-hosting on the new architecture.
Re:How can the Linux kernel compile itself?
by PeterisP on Tuesday September 17, 2013 @06:59PM
(#6402459)
For any practical purposes, it is best to simply use the full complex software stack on another computer to generate binary code meant for that new architecture.
If we'd really need to bootstrap a compiler from scratch, then a possibility would be to hand-code a Forth system, and then find/modify/make an assembler written in forth; then write a compiler for some small language (scheme?) in that assembler, and then from scheme you can easily get to some C compiler that can compile the full GCC.
Re:How can the Linux kernel compile itself?
by seclorum on Wednesday September 18, 2013 @05:05AM
(#6404283)
>Can a very minimal gcc be written which would be capable of bootstrapping the entire gcc compiler?
Yes. In fact, this is how gcc is built - in stages. First stage: build a compiler that can build the compiler. Recompile the compiler with the newly built compiler. Do it again. Repeat until completion. ;)
Re:How can the Linux kernel compile itself?
by OnACoffeeBreak on Tuesday September 17, 2013 @12:02PM
(#6400047)
Another interesting thing to remember is that an embedded operating system (Android phone OS and software, for example) is likely being built on a PC set up with a cross-compile environment and not directly on the embedded hardware.
Short answer: it can't. Each software system relies for its own existence on a predecessor, usually one of smaller scope. Sort of like biology and evolution.
When I wrote Apple Writer in 1978, there were no high-level development tools available, just a primitive assembler built into the Apple II BIOS. By making direct hand entries, I used the BIOS assembler to write a better, faster assembler that supported alphabetic labels. Then I used that assembler to write a better one, and eventually I had a development tool suitable for writing applications.
Linux development has relied on a similar bootstrap process, on a larger scale.
Thanks for this suggestion! I write a lot of articles, but I haven't written much about that period, possibly because other people were writing it for me. But I think it might be interesting to write about that time, how we created software before we had access to high-level languages.
In the meantime, here's an old piece I wrote in those days, even though not quite what you have in mind:
http://www.atariarchives.org/deli/cottage_computer_programmi...
http://cm.bell-labs.com/who/ken/trust.html
Point being: both relate to compiling compilers.
That's not what a kernel does...
"The kernel is written in C, however how did the kernel get compiled without a compiler installed?"
And...
"If the C compiler is installed on my machine before the kernel is compiled, how can the compiler itself get compiled without a compiler installed?"
The chicken or egg problem shows that at some point you lose the connection to the predecessor and the process goes around in endless circles.
Then you might wonder how it ever came to be when the first thing depends on the second and the second depends on the first.
No, this isn't so. It's a myth about biology that's repeated by people who don't understand natural selection. An egg is a chicken's way to make an insurance salesman.
Ultimately you don't need life to create life, and ultimately you don't need a computer program to build a computer program. There are other methods, even though we don't see those methods executed today.
The answer is that somebody punched in a program from a computer's console, probably somewhere in the early 1960s, but possibly earlier. That program was used to make it easier to enter other programs, etc.
Given that copying working code is so simple, chances are that most of the currently running hardware can be traced back to a few persons entering a boot loader by hand.
That might even be one person. For example, Windows NT was initially developed for Intel's i860 (http://en.wikipedia.org/wiki/Windows_NT#Development), so chances are that it was bootstrapped from a Unix machine. Because of that, it may trace back to the same initial boot loader as, for example, Mac OS X.
It seems like it would be relevant if a new architecture was developed at least.
If we'd really need to bootstrap a compiler from scratch, then a possibility would be to hand-code a Forth system, and then find/modify/make an assembler written in forth; then write a compiler for some small language (scheme?) in that assembler, and then from scheme you can easily get to some C compiler that can compile the full GCC.
Yes. In fact, this is how gcc is built - in stages. First stage: build a compiler that can build the compiler. Recompile the compiler with the newly built compiler. Do it again. Repeat until completion. ;)