One can bootstrap a system with a source-based package manager too. Here[0] is the process for bootstrapping a new architecture on Exherbo Linux. Note that the package manager is built on the host system and installs to the new system.

Exherbo supports cross-compiling with multiple concurrent architectures (both called 'multiarch') and installing into non-host environments.

A lot of other cool stuff, but the biggest point is that one can do what this blog describes with dependency management taken care of. The other point is that simply by managing the special 'system' set, one can pare down the base system. The OPTIONS flag can reduce system size by minimizing unneeded features (similar to Gentoo's USE).

[0]: http://exherbo.org/docs/bootstrapping.html

Can somebody explain to me the obsession with static linking? Do these people not realize that performance gains from shared memory between disparate applications hinges on dynamic linking mmap'd libraries and COWing shared memory sections to private on demand, reduces duplicate code, standardizes operations, and simplifies debugging? Has the world gone mad?!

Most of the applications you will have running at any given moment probably have few libraries in common. A typical linker will only include in the final executable any functions you require from your dependency as well, which can dramatically reduce the footprint of a piece of software (when considered in combination with its dependencies). Statically linked binaries also load faster and don't break when you have a botched update.

My distro does support dynamic linking, but prefers static. In practice the RAM footprint and disk footprint is much, much smaller than any dynamically linked distro I've tried. Avoiding GNU also helps a lot to reduce bloat.

Static linking also makes agunix more portable. Stuff like this is possible: https://sr.ht/5HTs.png

I have 878 unique libraries shared between currently running processes on my Linux laptop at this moment, with the amount of processes sharing a library ranging from 2 processes to 820. Most may have few in common, but there are a lot of them, and the mean is 62, mode is 4.

As far as the botched update argument: When does anybody botch an update of a shared library? When they're forcing something to install, or they don't know what they're doing. On the other hand, mis-matches in applications using different versions of the same code causes silent, unexpected, difficult to debug errors.

I'd also be interested in seeing actual "RAM and disk footprint" numbers as a comparison for the same applications and libraries, because that doesn't really make sense. Even if applications only copy a couple small functions each, they add up, whereas the shared library only has one copy.

I can't speak to the speed differences (again, I would like to see numbers) but certainly different versions could lead to competing cache and code path hits.

And a static application has nothing to do with portability. Just because I can run a Windows 95 application in Windows 10 doesn't mean it's going to work right, and just because FreeBSD has a Linux binary compatibility layer doesn't mean the app is going to work. Portability requires making it portable first. You can copy shared libraries with a dynamic executable and run it as reliably as a static app.

>I have 878 unique shared libraries shared between currently running processes on my Linux laptop at this moment, with the amount of processes sharing a library ranging from 2 processes to 820. Most may have few in common, but there are a lot of them, and the mean is 62, mode is 4.

Good god, this tells half the story. An agunix install booted to a shell will typically only have 3 processes running: runit, a getty, and your shell. I don't think I've ever had more than 50 processes running on my agunix system at any given moment.

>As far as the botched update argument: When does anybody botch an update of a shared library? When they're forcing something to install, or they don't know what they're doing.

For any number of reasons. This has happened to me dozens of times in the past year. You also run into issues with any software you compiled yourself that aren't managed by the package manager (or at least not supported by your distro). You know what's better? An executable that works on any Linux system and will continue to work for the next 20 years without any changes. How about running two executables that need different versions of a library?

>On the other hand, mis-matches in applications using different versions of the same code causes silent, unexpected, difficult to debug errors.

I haven't run into this yet. Have you?

>I'd also be interested in seeing actual "RAM and disk footprint" numbers as a comparison for the same applications and libraries, because that doesn't really make sense. Even if applications only copy a couple small functions each, they add up, whereas the shared library only has one copy.

My laptop is currently running runit, 8 gettys, sshd, wpa_supplicant, crond, udevd, and dhcpcd. It's using 60 MiB of RAM. So far as disk footprint is concerned, see package sizes at https://repo.agunix.org/. Base system is ~350 MiB on disk and there's still room for improvement.

From the agunix home page:

>Our first concern is to build a lightweight POSIX environment - a familiar design for users familiar with other Linux distributions is not our goal.

It's better to think of this as a new operating system entirely than as a new Linux distro.

It sounds like you're running a hobby Linux distribution. I have nothing against that, but please don't make it seem like somehow it is superior just because it is different. I used to make Linux distributions that could fit on a floppy disk, but I wouldn't brag about how fantastically small it is and how that obviously runs much better.

If you are compiling source code by hand, you are circumventing the entire distribution. Do not expect it to work. Just because it may work a lot of the time for you does not mean you should expect it to. If updating a shared library broke something, it's because you didn't know what you were doing. This does not mean shared libraries are bad. In fact, a great feature of shared libraries is the ability to update thousands of applications by simply changing one object file. LD_PRELOAD and many other features are incredibly handy, and don't work with statically compiled apps.

Yes, I have run into dependency incompatibility bugs. It happens when expected behavior in library code changes. If an application was not written expecting that behavior, then when it interacts with another application, it may have a conflict. The common places it will crop up are IPC, protocol, and file format changes, but also locking, support for optional functions, infrequently used features, etc. It can also happen from updating software's configuration in a way that is incompatible with some older version of some software that was using it, even though some other software supports the newer functionality. If you were around during the Linux desktop wars this was common. It happens less often with distributed applications since they mostly use APIs now, but some APIs suck balls and don't have versioning, and people will change the functionality of some part of the API that old code was depending on somewhere.

Copy an application from 5 years ago into a new system, and copy all the shared libraries into the new system too, and it should run, just like a static binary would run. And it will then immediately die, because all the other dependencies that don't have anything to do with linking weren't there, like kernel feature or modules changes, missing or changed non-library files, protocol changes, missing/incompatible daemons, abandoned deps, security paradigm shifts, etc etc.

Honestly, tracking and updating multiple copies of software with different dependencies is not difficult. We were doing that a decade ago at scale (tens of thousands of servers, thousands of individual software packages). People just don't like dealing with reality. So instead of dealing with the fact that they are doing complex things, they pick up their mess and throw it into a spare room. Look, the living room is clean now! Let's hope nobody looks in the other room.

Statically linked binaries deploy easily, deploy faster, start faster, don't depend on libraries or even on a filesystem hierarchy standard, because they don't require hardcoded elf interpreter and so on.

these are very desirable properties for binaries that are meant to be deployed. makes no sense whatsoever for distribution binaries though.

Up to a certain point (~64GB) I'd rather buy more ram than deal with any sort of problems of mismatched dependencies.

Think of it like docker without the network overhead, using the os kernel as kubernetes.

Think of it like webworkers with the kernel as the js runtime.

Ram is cheap. Complexity is expensive.

Multiple different versions of the same software running in the same environment increases complexity. Ram is cheap, loading code into ram wastes cycles and following new or different code paths trashes cpu cache and recent code paths.

So far I see only the opposite. Inability to be flexible about multiple versions adds a lot of unnecessary complexity.

dependency management is the main reason I love statically linked binaries.

Without static linking, getting the right ecosystem of libraries installed - especially when libraries depend on other libraries is a nightmare for any moderately complex software.

With static linking, scp over a binary and you're done.

That is a nice feature, but if code is only 1% of your process image, it doesn't matter.

Could someone elaborate on the "cons" the article mentions? I don't totally understand why those would be problems.

I originally went into more depth, but replaced it with a small summary of my motivations. The article is more about making a distro than about agunix itself. See the website for some information about agunix.

Service, configuration and package management are not solved problems in any distro. It's only normal to list systemd and apt as cons.

apt has significant limitations vs. a more flexible package manager. I would never use an apt-based distribution; I would use a Paludis-based distribution: see its feature list for many of the reasons why.

Bloat is a concern because I don't want unnecessary system processes running, nor do I want cruft installed that can be done without. Gentoo is good at minimizing cruft, and others in the same spirit as well.

Can't help you with systemd and GNU, as I am comfortable with both.

The writer seems more like a Gentoo or Exherbo or Arch user than any of the more popular distros. In other words, one of the people who used to use Linux when it wasn't as mainstream.

They're mostly matters of personal preference or opinion rather than real issues.

What's so wrong about GNU?

This is really a case of "If you have to ask, you'll never know."

That's not meant to be derogatory, just that if you don't have a problem with GNU, you won't have a problem with GNU, because there's nothing wrong with it, it's jut a matter of opinion (usually about the GPL).

There are some areas where GNU (particularly glibc) is somewhat bloaty in the code department compared to other implementations. That can, sometimes, matter, but I agree it tends to be a question of taste and not one of morals.

Allright! The article nor the FAQ page didn't mention why it was hated so much haha.

Thank for the answer!

I updated the FQA with some stuff discussed in these comments: https://agunix.org/fqa

Thanks! BTW, It seems to be a really tedious work, I like the result :)

glibc and buildtools can be a bit of a pain to work with. Maybe he was exposed to that during cross-compilation.

This is really cool, Drew. I can get behind a lot of the motivations here. I didn't realize clang can't compile Linux. Doas is awesome on OpenBSD and I've been using its port on FreeBSD too. I'm glad to see the Linux port getting more use too.

The coolest part about NetBSD for me right now is that you can cross-compile it on other OSs. That's quite a feat. However, OpenBSD has my heart for simplicity of core code. It has become my go-to reference for generic userland and kernel questions.

I'll have to follow the guide for installing this sometime. Some great stuff to learn here. Overall, keep up the awesome work. Seriously.

My 2012 MBP is getting a bit long in the tooth, and had several software-ish issues that were totally unfixable. As a long time Linux user, I decided to switch back, and purchased a powerful laptop to replace it. I did a similar survey of distros, and haven't been happy with any of them.

Ubuntu worked fairly well in a vanilla configuration, but as a developer, I prefer a more powerful and customizable desktop UI. My preferred window manager is awesomewm, but trying to get it to integrate with Unity or Gnome-shell is pretty much impossible. If I run just bare awesome, I lose all the power management & audio niceties. I can probably set it all back up, but documentation on how to do so is slim. Additionally, installing newer software than what is provided by apt is a giant pain. You basically have to hope the vendor provides their own apt repository for you to use, and that they've updated it for Zesty.

Arch linux was by far my favorite before I switched to OSX ~10 years ago. It always has the latest software available in core. Additionally, AUR provides betas and even more esoteric software. Anything I come across here or on reddit probably already has an AUR package for it, and in the rare case it does not, it is trivial to write a PKGBUILD for it. However, installing Arch is way harder than it used to be, particularly on laptops. There's so much variety and complexity in hardware now, you could spend a month getting everything set up and it still would not be satisfactory. Power management, sound, full-disk-encryption, webcam, display hot-plugging, hardware sensors...

I used Fedora for quite awhile on a desktop before I got this laptop, and really enjoyed it. However, the latest Fedora uses wayland for everything, which doesn't work with the closed nvidia drivers, and nouveau on the 1070 in this laptop basically doesn't work. However, `dnf` is 10x better than `apt`, and I found it much easier to find copr repositories for updated software. Maybe I'll revisit once wayland and nvidia work together.

I wish there was a distro half-way in between Arch and Ubuntu/Fedora. A nice installer that autodetects and sets up hardware automatically, but still easy to hack on. The problem with the former, though, is you pretty much need the resources of Canonical or Red Hat to get it done.

Also, for the record, I love systemd. I've been using unixes for 20 years, and never managed to grok init scripts that I was comfortable creating or modifying them. Systemd (like Upstart and Launchd before it) is a huge improvement over that mess.

Cons include ... systemd. The author doesn't elaborate how it's a con, same as some other examples.

It's clear the author thinks it's a con, but I don't see why this article should justify that, it's a pretty unrelated topic to the one at hand.

The tone of the article suggests it's something obvious and universal, which it's not. So there should have been some elaboration, or at least something like "for my purposes".

Most of that sounds quite debatable and a matter of taste. I don't find those arguments objective enough. Calling anything they don't like "bloated" doesn't sound right.

Have you ever dug into the internals of GNU tools and compared them to other tools in the same niche? It's self-evident.

The one that stumped me was GNU. Do they mean the GPL? GNU userspace? gcc?

Yeah, that wasn't clear.

While normally I would agree, if the goal is to run your own distro for personal use, you want it to to fit your own needs and abilities as closely as possibly. If you consider systemd to be annoying, or bloated, or complicated, since you're the main and/or only user, that's an extremely valid reason to not use it in that case.

The reasoning for a large public distro is different, because most users might not ever even know what they are interacting with, or only interact with it through a few layers, so other features are more important (and in my opinion, where systemd really has some benefits).

And that may well be the biggest split that has come within the Linux communities in recent years.

Across time home grown minimalism and large name distros have largely coexisted in piece. This because large name distros where at its core an expanded home grown minimalist setup.

But systemd is shifting all that, because it is clearly built from the ground up with large name distro wants/needs in mind (or maybe it is the wants/needs of big name DEs, with Gnome at the front, and distros have to begrudgingly accept systemd as they want to ship with a big name DE as their default UI), and little else.

And as time goes on, it will be harder and harder to find projects that do not expect some part of systemd to be present. And given how systemd is maintained it will be hard to use just those parts and nothing else.

More and more systemd seems like a massive case of the tail wagging the dog.

At some point I think we just have to accept that, like most automobiles, all the things we want come with a complexity cost, and eventually the complexity is too much for the average person, and then too much for even most exceptional people. Compartmentalization helps this somewhat, even if it initially looks like the opposite.

You can still build a hot rod without too much advanced knowledge. Want to include anti-lock brakes? What about traction control? What about variable valve timing? At some point even the savants will start acquiring whole systems to drop in as needed, as needed. Systemd is to sysinit what a brand new aluminum or magnesium alloy modern engine is to a 70's cast iron engine. It's not something you want to mess with too much, and it has a bunch of inputs and outputs you probably don't care about, but it's designed for what it's used for. If I was buying a new car, I want the modern engine unless I expect to need to do a bunch of maintenance myself.

This is the normal progression technology takes. It's fine to still run your own distro, just as it's fine to build your own car, but let's not delude ourselves into thinking the custom distro not the hot rod are just as good general purpose devices, even if they may well be better for a specific subset of things (and that subset may well fit what you care about).

Wouldn't you expect more decoupling and decentralization with growing complexity instead of a single engine that does everything? It's not like we are constrained by size or weight of an automobile, we can simply agree on some IPC protocol and let everyone have their own managing daemons completely isolated from the rest of the system, that are started and managed by other managing daemons and so on. In fact, this is the only way to make it reliable, otherwise errors in centralized components will affect everything.

I don't think it is one thing doing everything. It may seem that way if you zoom in to a small portion of what the system does, but it's a fairly contained system that has some parts that extend beyond what you would normally consider it's boundaries to achieve specific goals.

The kernel handles hardware abstractions and provides a base interface. Systemd handles system boot and service management.The windowing system (including window manager) handles GUI interaction. Whether you would or could draw the definitions elsewhere doesn't really matter, as the systems are in enough flux (especially recently) that they aren't very static. As features are needed, they shift between those systems (or or even others) to find a place where there are resources to implement it, where it's feasible to to, and where it's efficient to do. Those aspects are weighted differently over time (especially resources available, which is developers with skill and time), so some features will swap between systems.

A well defined API between them would be nice, but that's not the type of development that's generally done in open source. It requires a lot more understanding, effort and time up front, and even where there's a will, there those resources don't always exist. Systemd tried to solve a lot of problems, and create new capabilities (or bring them to linux at least). Expecting everything to both have not only have been thought about, but planned, for, tested and accounted for prior to it starting is a tall order. Even if ther had been an exceptional amount of effort put into that, it's likely that other systems (such as the window system or the kernel) would change to offer new capabilities or want to offload some responsibility that they saw as better handled elsewhere. This is where the metaphor breaks down. A car is generally designed in whole by one group (or the systems they integrate are complete and finished), and that doesn't really exist in Linux, since everything is always in flux.

To be fair, I also hate every other init system.

Essentially a perfectionist who can't settle, reminds me of myself.

I've made a good experience with Arch Linux, and since I didn't care much about the init system it worked quite well for me. Sadly the hardware didn't so I ended up in another world again. Happens I guess.

What would you like to see in an init system/service manager?

runit but with better code, basically.

There are plenty of workalikes available.

* http://jdebp.eu./FGA/daemontools-family.html

That said, I disagree with your premise that runit has bad code.

Thanks for the link! I'll have a look at these.

>That said, I disagree with your premise that runit has bad code.

Have you seen it? Bad code may be subjective, but it certainly doesn't come close to my standards.

Of course I have seen it. A lot of it is the same free Bernstein libraries that have been used by Felix von Leitner, Paul Jarc, Laurent Bercot, et al., with some of the same C1999 modernizations; and that inspired various similar libraries.

* http://www.fefe.de/djb/

* http://www.skarnet.org/software/skalibs/libstddjb/

* http://untroubled.org/bglibs/

* http://www.fefe.de/libowfat/

Systemd sucks, so it's not a troll.

Yet another anti-GNU, anti-systemd article.

For folks interested in reading about how distros bootstrap themselves, the GNU Guix manual has some good details:

https://www.gnu.org/software/guix/manual/html_node/Bootstrap...

No, it's an article about making a new Linux distro, with a brief summary of my motivations. Someday I might write an anti-GNU, anti-systemd article, but this is hardly it.

If you are anti-GNU and anti-systemd, I am highly skeptical of your design choices.