LFS helped me a lot. It taught me how to compile packages manually, and how GNU/Linux components fit together.
Armed with that knowledge, I could do things others could not do:
- Make applications that bundle/automatically download their dependencies, compile them with custom flags (static linking)
- Diagnose build errors buried in deep dependencies when using vcpkg
- Compile programs for esotheric systems: e.g: Intel MacOS X 10.5, an obsolete system noone else on earth is using except the company I worked at
- Create application targetting obsolete systems but using the latest version of its dependencies
- Compile recent programs on esotheric systems, by bootstraping a newer gcc on the newer system, patching as necessary
- Create my own stripped down distro, tuned to my specifications
- Use that stripped distro as a docker container
- Run recent linux on obsolete hardware no longer supported by mainstream distros
- Reformat a cloud dedicated server without physical access by first bootstraping a minimal distro on a ramdisk, unmounting the main drive, partitioning that drive, and installing the new distro on that drive. Reboot and pray.
- Build a cross-compiler for targetting a freestanding environment
- Build latest KDE/XFCE for debian stable. No more "but debian ships with outdated software!!11"
- Build applications on a distro they were not designed for (Zimbra OSE on debian)
I think you missed the main argument though - LFS isn't quite as good as it could be. That is a correct statement that was made.
LFS is about the muscle memory; the documentation; the cohesiveness.
I always recommend it to people and consider it foundational to my understanding of How Stuff Works.
Whether that amounts to a hill of beans in the AI world remains to be seen.
Fundamental things like how processes are scheduled are not explained by LFS.
Again I disagree. Way too much time is spent building software than actual practical OS development.
I certainly do not miss compiling X11!
Right. I used to compile many open source packages from source on many flavors of Unix back in the day, for years. Perl, Python, MySQL, many others. Miscellaneous utilities. Esoteric languages. Even Oracle, once, on a Unix server, right at a large manufacturing company site. Even building Unix kernels from source was common on some Unix versions. And not just for a regular install. Many times, (re)building Unix kernels from source, was the only way to get some software packages or hardware devices or peripherals, like printers or terminal controllers or even hard disk controllers, to work on the system. Typically you would edit some config files or settings files manually, change values of params, or in some cases use a TUI to do that, and then rebuild via a makefile. You had to save the old kernel in case the new one didn't work, and if so, lather, rinse, repeat, till everything did work.
And that practice was nothing special. A lot if people did it routinely. Most, in fact, I'd say. Talking about devs and sysadmins here, not end users, although some of the latter probably did it too.
Many apps, tools and utilities came as source code, often in a .tar or .tar.gz file. You would untar or gunzip it into a new directory, and do your stuff, running makefiles, other shell commands, setting env. vars., etc.
All this for running on embedded Linux with 32MB flash drive running from read-only volume powering two kiosk displays.
Sounds like a speedrun of the tutorial. How did you approach packaging then and what system did you use to compile everything?
I did the tutorial in the late 2000s I think and it took me 4-6 months if I remember correctly. At the time I wasn't completely new to Linux but setting up things like the boot loader was quite tricky for me.
For me the main motivation was getting a deeper understanding, being able to fine tune and customize my system. Of course I didn't go blindly through the tutorial but only did steps that I understood. (In case something goes wrong) Also I started to use (x)stow for everything.
Requiring four months seems as if it was a side project, which is of course totally fine, but for any main task it really should not take more than a few days, even on a semi-slow computer (well, depends on how slow).
That actually makes perfect sense. If you did the bootloader first, how are you going to build it? How do you know what to boot from which filesystem on what device? By the time you've got your root filesystem complete, it's easy to tell.
If you do things step by step incrementally, you keep having to go further back in the process. Starting over repeatedly gets tedious, boring and frustrating. And it'll take ages to finish the complete OS.
IMO the guy is right - LFS could be way better. But it has this current structure and does not deviate from it. That's the bad part.
> If you do things step by step incrementally, you keep having to go further back in the process. Starting over repeatedly gets tedious, boring and frustrating. And it'll take ages to finish the complete OS.
Pray tell how that is necessary for the bootloader? The chain used in LFS isn't the only one that makes sense or is required. IMO it would indeed be better to have a system you boot into as quickly as possible, on real hardware, then continue from there.
If you like the idea of everything being purpose built for your computer, then after you do LFS you should switch to gentoo.
And you dont need to boot anything, just a busy loop after a log "Hello World" is enough, if you feel fancy, maybe make a simple shell ?
If I installed the bootloader first, I'd have to remember to run dracut after the system was installed. Or I install it last, create the init at the same time and make a habit of it.
So at points you want to dive deeper, you do — and go off the beaten path.
It requires a desire not just to understand how bits and pieces fit together, but how you can make them work even better for yourself and what is and is not customizable?
Arguably, even if building it today is easier — back when I did it in 2000, you sometimes struggled to find where the canonical source code of the package is, not to mention compilation errors, linking errors, etc. Only then did you get to configuring the init RC system, choosing between them (no systemd yet), etc.
But a requirement is — IMO — that you are strongly opinionated about how a system should serve you, and look for opportunities to tune it.
I know a french cookbook[0] which contains not only recipes for dishes, but also for some of the intermediary ingredients, like stocks, that can now be bought at the supermarket. I feel that I’ve learned a lot by following those recipes, be it simply in the structure of french cuisine. And to me, LFS is a cookbook in a similar sense.
[0]: https://www.ferrandi-paris.com/fr/le-grand-cours-de-cuisine-... in French, sorry!
How would it compare, in your experience, to other (modern) cooking books with a similar scope?
Most educational resources are what- and how-based, only the best ones are primarily why-based, what Wirth once wrote in the preface to some Pascal book (I paraphrase) 'discussions are motivated, so that the learner gets coverage and understanding, instead of memorization'.
Principles can be studied from “Operating Systems: Three Easy Pieces“ book - https://pages.cs.wisc.edu/~remzi/OSTEP/
Rob Love wrote the book “Linux Kernel Development”. https://rlove.org/
And then continue with reading LFS.
It was like automated LFS from git
FreeBSD’s Makefile system is also in this ballpark
Too bad Cross-LFS is no longer a thing. Then again, with all these microcontrollers and ARM and RISC-V systems today, cross-compiling is no longer as magic (and tricky) as it was back then...
Shameless plug: I started a series of blog posts exactly because this. https://serversfor.dev/linux-inside-out/
Unfortunately due to some cervical spine issues and a surgery I wasn't able to finish it, but I am planning to pick it up and continue.
It’s on my list of things to do when I have some free time
I mean, eh... last time I checked, this was Hacker news. Yesterday, there was an Ask HN topic on how to enable dark mode on HN. Is this the new hacker culture? Complaining that the resources merely point in some direction instead of specifying details?
Exactly my experience. Back in the day when all I knew was Windows and a couple of BSDs I wanted to get into Linux, because it seemed like it was going to become very relevant in the next decades. After years of slacking I thought LFS would probably be the ideal choice. Turned out it wasn't, at least for me. I ended up "learning Linux" with distros like Gentoo, Arch and later even Ubuntu, and by just replacing my production systems with Linux, which were still running on OpenBSD mostly. I have revisited LFS a couple of years ago and honestly didn't take much from it. I get the appeal, but it's not for everyone.
Bare LFS is not enough (I run my own): you need a kind of userland (above glibc) multi-version system which some kind of "atomic-ish" switching (always have a stable SSH running in case something goes wrong, better than unplugging the system disk and fix it on another computer).
For linux, same thing, with a kind of flip-flop-ing for updates/fix/etc.
The insight that gives you will probably scare you: the current "open source" stack is an abomination (the worst is the SDKs I think).
That's why we need _LEAN_ open source, and that includes the SDKs.
I promised myself not to work for any of those anymore :)
Modifications and changes must be weighted very carefully. For instance "removing" (including in the SDK) is usually a less worse modifications than the others and it decreases the global technical size and complexity/number of layers of the stack/SDK (which is top priority in "security"), namely most of the time but not everytime a good thing.
Because, on most complex software (open source or not), a "believed" benign modification can be disastrous. It is even worst if close to bare metal involving complex hardware (even with massive QA[testing]).
That said, port trees usually have devel-* sections. Does that count?
You'll see also a good amount of more than questionable code generators. And when you will have a look at some _GNU_ makefiles of big corner-stone projects, you will question the sanity of their authors (the "makefile" itself is often, but not all the time, very questionable too nowdays). Everything was with a good intent, turned sour and painful on the long run.
On our modern computers, we should have "one-compilation-units" for most, if not all binary products (exes or shared libs) anyways. "Same language" binary static linking in the open source world should not be a thing anymore (have a look at those 'header-only' projects).
The invisible backdoor injectors (namely compilers) are a problem, for many reasons that most know here (barrier to implementation of real-life alternatives, [open source] developer lock-in, etc). The more complex is the syntax of a computer language, the worst it is (on c++ syntax complexity scale, or with a heavy/intrusive required runtime like many "new" languages do shove down your throat).
If you don't see any issue, get cproc/qbe C compiler (and the binutils gas assembler), then you compile linux (with a brutal and linear shell script, let's say for a real life x86_64 desktop) and we'll talk again after, ok? And once you succeed, we'll see how it holds in 7-10 years (the time ISO takes to break in a non pertinent way many things).
If I'd want to roll my own distro again, I'll have a look at an old dir with lots of scripts and patches where I tried doing just that. (I bet almost none of it works with current tarball versions.)
I think I'll retire with NetBSD. Keep up the good work though, someone ought to highlight the insanity of build tools building build tools.
For this major reason, computer languages with a "simple" syntax and "alternative" real-life compilers should be favored (or you know it is reasonable to build one from scratch). The current _LESS WORSE_ alternative is C and similar. But this is far from a silver bullet. I think our only way out, namely getting rid of all this toxicity is assembly without abuse of any macro-preprocessor, that's why worldwide non-IP locked ISA standard like RISC-V are so much important.
If you really want to start over from scratch, some form of assembly is indeed required :) And the first assembler would have to be assembled by hand. I'm not sure if riscv is useful for that, I'd try finding the oldest m68k still serviceable. And the sooner you get to C, the better. Starting from scratch is one thing, doing it all in asm takes more than a lifetime. And from C to various interpreters, of course. Most programs can be written in a scripting language and have fewer bugs than if they were written in more verbose languages.
But where I do agree: I presented the "assembly saving the world from big tech toxicity" would come with very high level scripting languages (probably a mess... but less toxic than what we have today), but with their interpreters/JIT "compilers" written is such assembly without abuse of any macro pre-processor.
I gave some thoughts about porting Bellard quickjs to pure hand written/tool assisted RISC-V assembly.
What I described is an example of build bootstrapping. There's a website about it, https://www.bootstrappable.org/
Some of Big Tech hates "small is beautiful" and stability in time then will fight hard with their $ for that not to happen (administration "corruption"). Complexity and size (including the SDK then the computer language syntax complexity) is their bread and butter to keep "control" and keep alternatives away.
For the "bootstrappable", it is the starting point of building from 'small and simple' toward big and complex, aka the SDK 'binary seed'. At the time anybody would expect a 'simple C compiler' with a simple linker and assembler buildable with that simple C compiler... but people at the gcc steering commitee and the usage of many C extensions without maintaining proper fallback to assembler source files (in kernels, hence linux) broke the chain of 'sanity' for good. Now you must have an ultra complex c++ compiler to "bootstrap" (~LFS), not to mention the required zillions of complex tools required by the "build systems". Lately, it seems AI coding may be the salvation here... How? AI seems to assist writting c++(insert you computer language with a complex syntax here) to simple C "transpilers" (somebody pretended to have done the same with a rust->simple C transpiler, was published here on HN). I have to say I have never used AI, since they are gated with the web engines of the whatwg cartel (I cannot even get a token to use some web API), I have never tested.
The conclusion: fixing the software stack to let sane small alternatives to 'exist' and be 'real life' is probably not going to be calm and quiet.
Re: bootstrapping, your computer, your OS, your choice. When I power on my laptop it starts grub, linux, a shell script, sysvinit, at the moment openrc then kicks in (I want to replace that one), and all that's left running is init, sysklogd, chronyd, cronie, and agetty.
And I don't think vibe coding is going to help much at all unless you want another templeos. I think it could do that.
That "minimal platform for interop" would be made of simple but able to do a good job enough and stable in time file formats and protocols. This minimal platform must be reasonably implemented with "Small Tech" (excluding de facto the whatwg cartel web engine and computer languages with ultra-complex languages or heavy runtime this is common sense).
That "minimal platform" should probably be made with subsets of current file formats and protocols. As I said, for instance a subset of PDF (with validation tools), mandatory handling of basic UTF-8 text files, no script/basic HTML for the web, brutally simple SMTP support (aka self-hosted, not paying for DNS, or email addresses with IPs literals, only the commands to send actually emails without fideling around), IRC bridge for chat, etc.
BTW, on the "assembly to save us", I was talking about very high level languages with interpreters/JITs coded directly in assembly (aka binary specifications), then Mr. Bellard quickjs, but also obviously python (but it seems its syntax is going amok which is bad omens)/lua/etc.
This "platform" should slow down a lot the endless enshitification cycle we are into.
If AI coding is that powerfull (cannot test it as its access is gated by the whatwg cartel web engines), it can probably help: like I said, since we are not alone, look at what the others like me did: they are using it to write transpilers from ultra-complex computer language syntaxes, I saw on HN this norvegian guy who is using AI coding to help write directly in 'human readable' assembly (here x86_64) the binary specifications of its own everyday software.
This plaform must be very rigid, but not completely, namely rational removal/restructuring could happen. Basically, implementation "from scratch" by "Small Tech" must always be reasonable.
Aside of all that, there is the ultimate hard truth: for much software developement, but not all, it is extremely hard to justify a permanent income and that honestly. Making all that very complicated to fit in any economy.
Code written in assembly is more error prone, more difficult to debug and harder to understand than higher level languages. After years of reading and writing code I know being able to reason with others about the source code of a given program is more important than what the actual language is, or any of the other purities programmers like to keep. Without readable source, the program either dies a slow death or becomes some kind of minefield. And besides, throwing out decades of development is such a waste of effort.
I don't understand your comment on permanent income and the economy. Sorry.
Bootloaders as we know them came after OSes, because the need of a bootloader is to boot something. Following your logic of building a piece only if needed, a bootloader should come after.
I like the recipe analogy and I actually think adding things only if needed will help cutting unuseful features and possibly to have different implementations for solutions you need.
No point in reinvent the whole thing, tho. Following your analogy, if all cakes need egg, there must be a reason even if we don't know it.
We already have means to test the new OS.
There is real point in writing a new one until the OS is complete, is there?
We already have a bootloader.
"Linux from scratch" involves Linux, not Libreboot, not Grub (just to mention instances of BIOS/UEFI and bootloader).
We already have means to test the new OS.
What I meant with my previous post is "newer" bootloader (which are actually pretty old, like grub), came after OSes passed a certain complexity.
Previously we technically had bootloaders, but they were interactive, mostly.
Am I misremembering?
Agree! imo this is the guiding principle that makes great tutorials great.
It’s just too much to take in at once. You start by building a cross compilation toolchain but… why? And you add the bootloader dead last, even though it’s the first thing you see when you boot a Linux machine?
I reckon a better approach would be to build it step by step, only adding new things once you’ve discovered a need for them. First you get the bootloader working, then you get it to actually load the initramfs and kernel, etc… you could even create your own extremely simple initramfs first before adding dracut, etc…
This gets you to a system you can boot and play with asap. And then you can slowly start to discover why you actually need all these things through exploration and experimentation.