> I think I understand the intuition behind that... in other words... coding an os kernel seems to have less entropy, less degrees-of-freedom, less subjectivity ... than frontend programming like painting GUI pixels and Javascript-framework-of-the-month.

I don't think that's the right takeaway. A decade ago I worked as an SAP consultant, and did some payroll projects in it. The display part of it was easy. All they wanted was a great big table display, the moral equivalent of a read-only Excel spreadsheet. No faffing about with CSS or Javascript or whatever.

No, the real complexity was the freaking business logic. Office workers, executives, contractors all had different rules for how their pay was calculated, there were all sorts of line items that applied to different classes of employee differently. The company had spent years building those reports by hand and there was a whole lot of organisational knowledge hidden in their heads, corner cases and exceptions all over the place. The sort of thing you can navigate in your head relatively easily if you've been doing it for years on end, but a newcomer has no chance of understanding - so a lot of iteration was involved in getting where we needed to be.

Put fiddly business logic and rapid iteration together, and you're burying yourself under a massive pile of spaghetti code in no time, flat. Trying to keep all of that sane both for myself and for whomever came after me was challenging in ways that none of the more systems-y work I've done since has ever replicated.

Yes. This is why the much-maligned (around here) category of “enterprise software” is so hard and why the results are often kind of shitty.

Perhaps another way to describe this intuition is that with operating systems, the developer is also the subject matter expert and the one setting the requirements. When you are the one setting requirements, those requirements are going to seem inherently less arbitrary than requirements set by someone else.

Many are the sects who pursue the Way, and countless their contradictions! Those who write accounting apps, in contrast, are just hella boned by uninteresting complexity.

In my experience OS kernels just make more sense than most front-end code, regardless of the kernel design. There may be different plans for each kernel, but there is a plan. Major things change less frequently, changes are better thought-through, and people think more about what they're doing. Probably this makes economic sense, and the costs of kernel breakage are that much more than the costs of a broken UI, but definitely the kernel's design is easier to make -- because it has a design, rather than ever-shifting chaos.

Exactly, working at a low level still means that “the programmer operates as a mediator between people having different ideas”. The different people are just hardware builders instead of library writers.

But I think that even with all of that, writing an OS is still much easier than an accounting package. The way forward is paved smooth by hundreds that have gone before. And by that I mean, hundreds of Unix clones have been made and open-sourced. Things like microkernels and NT-style I/O architecture are esoteric things that you only see in the embedded world (which Google seems to be limiting Fuchsia to at the moment) and enterprise-oriented operating systems (like Windows, which home users only use out of a weird quirk of history).

A consequence of AT&T originally not being allowed to sell UNIX and the existence of the Annotated UNIX book.

Without it, UNIX would have been as esoteric as anything else and most likely would have died instead of sprung multiple clones.

Genuinely interested, do you have some pointers about what you mean by NT-style IO? How would you characterize it?

I was referring to the "David Cutler (designer of VMS & Windows NT) criticizing UNIX i/o architecture" comment above, which apparently refers to Dave Cutler's criticism of Unix as "Get a byte, get a byte, get a byte byte byte", which seems to me to refer to the Unix/Linux style of programming (in which programmers overuse things like getch() and putchar()). Whew.

But. There's another, much bigger difference between Unix and VAX/VMS/WinNT: The way I/O operations are handled in the kernel itself. In Unix systems, the kernel just passes I/O operations directly to the driver code, which immediately tries to do the operations on hardware. It's a very microcontroller-ish view of the world. WinNT treats I/O operations as sort of a "packet" which encodes the operation to be performed. That packet passes down through the layers of the kernel, including 3rd party modules and eventually drivers, until the operation is performed on hardware. Then, another packet percolates back up the stack to your app with the result.

Things like antivirus, dropbox, etc. can insert themselves into the stack and intercept/modify/reroute I/O operations and results globally.

I think that Linux probably has something like this, or you can probably cobble it together by using async I/O, io_uring, and/or eBPF.

One thing to remember about it is how UNIX's I/O model is designed for 64kB total of memory space on simplistic mid-range (at best) computer. While trying to fit in concepts from 36bit Multics machine whose I/O primitive was equivalent of mmap().

POSIX is ask first (can I do I/O?) then do, while NT is do, then ask (is it done?).

TempleOS is the one pure operating system created without any debates. RIP Terry A. Davis.

You mean, Terry won all of the debates ;)

The only OS endorsed by the Holy C.

Exactly.

Dig around in C land and you’ll find 100s of opinions, even more libs and toolkits that came and went out of fashion.

Web stacks are “computing” to industry right now. In the 80s and 90s it was OS and database.

If low-level had the old economic attention, we’d see the same activity and mess as we did in the 80s and 90s. Everyone has a compiler to tell you about, and a build system DSL that’s just right for your problem!

But big corp largely controls that and so a new layer of abstraction was added via web, and we’re seeing a shift now to less UI, and machine driven decision making that just gives us the next step in the recipe.

Frankly I don’t mind technology for navigating human society; buying stuff and literal navigation.

I still unplug with musical instruments and the forest.

The addictive applications we’ve built in technology; games, social and multimedia, got really banal for me a long time ago. If I hear things like “Itsa me, Mario” or something about Halo I recoil at the cringe.

I guess I don’t want my psychedelic noise wrapped up in branding and consumerism.