Considering that this is exactly how Linux was born (just a hobby project for fun), I wouldn't assume so fast that it's useless. Moreover, you don't need to justify yourself if you just want to have fun !
I agree they shouldn't have to justify themselves, but it is handy to preempt that.
In fact in this instance I think it's a little disingenuous to quote Linux and say it could happen again. The industry is totally different now. There's much more competition than there was when Linux was released and that competition is much more mature too. Plus the bar for a production-quality kernel is a lot higher than it was when Linux was released.
Seems apropos to me, given the fact that a Linux ABI compatible hobby project is under discussion - and that everyone here is familiar with the famous Usenet announcement.
> The industry is totally different now. There's much more competition than there was when...
I wonder how you define "competition"? Because there were way more operating systems in use then, and the industry was far more fractured. Fractured in a way that was meaningful - not like today where you can spin up a VM and be productive in short order, thanks to the significant lack of distinguishing difference. That is the really interesting thing about these hobby projects - they introduce possibilities that are either completely ignored by organizations suffering from inertia constraints, or can't be mimicked because they're diametrically opposed to present designs.
There is way more competition now:
+ Linux (countless distributions)
+ FreeBSD
+ OpenBSD
+ NetBSD
+ DragonflyBSD
+ HardenedBSD
+ Darwin
+ Minix 3 (which wasn't free when Linux was released)
+ Illumos
+ OpenIndiana
+ Nexenta OS
+ SmartOS
+ ...and many others based off OpenSolaris / Illumos
Don't conflate standardisation of the industry with a lack of options. More options do exist today and are in use (eg some games consoles run FreeBSD, Netflix uses BSD, Nexenta is used in some enterprise storage solutions, Darwin may not be used in any free capacity but macOS is clearly used heavily by HN readers, and so on and so forth).
Moreover, I've used FreeBSD, Solaris, OpenSolaris, Nexenta and OpenBSD on production systems over the last 10 years (and the list gets more esoteric if we look past 10 years). So just because you might exist in a Linux-only ecosystem it doesn't mean that's the case for the entire IT industry.
> So just because you might exist in a Linux-only...
Obnoxiously presumptuous.
lol it's not nonsense. Every item I've listed offers something unique.
> Just look at the last 5 items... seriously - all these Illumos derivatives are distinct operating systems in competition with one another?
Actually the differences in the Illumos derivatives are quite fundamental:
- Nexentra is aimed at being an enterprise storage solution with Debian user land. That's massively unlike Illumos
- Smart OS is designed to be a smarter virtualisation and containerisation host OS. It even bundles Linux's KVM virtualization. It's not intended to be run as a desktop platform
- OpenIndiana and Illumos are probably the most similar, however OpenIndiana aims to be more true to OpenSolaris while Illumos aims to be more of a hybrid upstream. So while they're both multi-paradigm (like how Debian can be a desktop or server OS) there are some major differences in their commit tree.
Honest question: how close are you to the Illumos projects? Or are you just an outsider looking in and making assumptions about equivalence based on your experience with Linux? I ask because Illumos forks are much more akin to BSD forks than they are Linux distros.
> Be serous, that list only further proves my point about the total lack of distinguishing difference in the present offerings - relative to the pre-linux environment.
Even if you want to pare down the list to upstreams, you still have half a dozen BSDs (I really hope you at least have enough experience with BSD to realise these aren't just respins like Linux distros), Illumos, Minix, and Darwin. Verses your point in the 90s which was basically non-free BSD, non-free SysV, non-free QNX, non-free Minix etc....you still have those options now PLUS the ones I've listed.
This is the problem with your argument. You're assuming the old choices have gone away, which they haven't...well, apart from SCO and Minix is now free. And in addition we have dozens of interesting new platforms, some of which I've exampled, too.
There's no way on Earth we have less choice now than in the 90s. We might have the industry largely standardising on a subset but that's an entirely different argument and it's only representative of the lowest common denominator doing common problems. However if you look slightly outside of the status quo and you'd see there is a lot of variety still happening in the industry. I know this first hand too -- as I've said in my previous post, I've worked in plenty of places that weren't just Linux shops :)
You then go on to characterize components grafted in from another OS as "unique". That is a laughably low bar, making every potential permutation of existing operating system components distinct offerings in competition for mindshare.
> Honest question...
Honest answer: you are running code I've written. I've been using ZFS in production for as long as one possibly can. My home file server has been SmartOS for many years, before that it was FreeBSD.
> I really hope you at least have enough experience with BSD to realise...
I'm keenly aware of their differences and commonality, having tracked the introduction of a terminal mode bug all they way back to a time when "Melbourne getty" was a thing.
> There's no way on Earth we have less choice now than in the 90s.
Well, I suppose that if you are including abandonware and dead product offerings in the list of competing operating systems...
> We might have the industry largely standardising on...
It seems like you are trying really hard to avoid using the more appropriate word for what has happened: "consolidating".
That's literally no different to how Unix evolved. You talk about diversity in the 90s when most of the platforms were being trolled by SCO for containing the same code base. Don't you see the hypocrisy of your argument there?
> Honest answer: you are running code I've written. I've been using ZFS in production for as long as one possibly can. My home file server has been SmartOS for many years, before that it was FreeBSD.
Then it seems very strange that you don't acknowledge the differences in current platforms while proclaiming that Unix was more diversified in the 90s (if it was that clear cut SCO wouldn't have been trolling everyone).
> Well, I suppose that if you are including abandonware and dead product offerings in the list of competing operating systems...
You're overstating things once again. :)
> It seems like you are trying really hard to avoid using the more appropriate word for what has happened: "consolidating".
No. Consolidating means the removal of options. Those options still exist they're just not as commonly used. Thus term I used of "standardisation" is more apt.
Look, I do understand the rose tinted glasses you're wearing. There are aspects of 90s era systems administration and development that I miss too. But I still feel you're way off the mark with your opinions here. In some places you are exaggerating a nuanced point to such an extent that as much as I'd love to cheer on for the "good old days of computing", your comments simply don't represent my experiences then nor now.
Uh, that is exactly my point. You know that you were just arguing that they were unique operating systems, right? You don't see how the comparison totally undermines that position?
> ...while proclaiming that Unix was more diversified in the 90s...
I'll never understand why anyone bothers fabricating strawmen in thread based mediums - I never made that claim, and that is plainly obvious to anybody who has the ability to scroll up. I have a hard time believing that you could, in good faith, read "there were way more operating systems in use then" and honestly think "oh, he is obviously only talking about Unix!"
> No. Consolidating means the removal of options.
Yes... you remember what the option was selecting from? I'll save you the trouble of manipulating your scroll wheel: "The industry is totally different now. There's much more competition... the bar for a production-quality kernel is a lot higher..."
You still wanna keep saying that stuff from before, about options?
I never said they were unique operating systems. I said we have more choice now than we ever with regards to Unix-like systems and I said they were distinct platforms, code bases, even upstreams. All of which is true. I have no interest in entering a philosophical question of what changes constitute a "unique operating system" however we can at least have a technical discussion about choice and build of the Unix-ecosystem.
> > ...while proclaiming that Unix was more diversified in the 90s...
> I'll never understand why anyone bothers fabricating strawmen in thread based mediums - I never made that claim, and that is plainly obvious to anybody who has the ability to scroll up.
All these comments you make of strawman arguments and now I can see that the original reason for our disagreement was that you replied arguing a different point to me from the outset!
I was only ever talking about Unix-like platforms (given we're talking about Linux ABI) and you misunderstood that post to think we were talking about operating systems in the broader sense. In fact the language I used should have made the scope explicit so I'm giving you the benefit of the doubt here that you weren't intentionally just pulling an asshole move of subtly changing the scope to win a different argument (which is the very definition of a straw man -- since we're already arguing about who's trying to pull a straw man)
> I have a hard time believing that you could, in good faith, read "there were way more operating systems in use then" and honestly think "oh, he is obviously only talking about Unix!"
I was though. You were the one who replied to my point where I was only talking about Unix so good faith would dictate you continue with the same scope and context as the conversation started out with. So yes, I did assume you were still only talking about Unix. Because that's what the conversation was about prior to you joining it.
-----
If we are to discuss your point then I agree there are fewer operating systems in general. And I agree that is a great loss. I'm also happy to talk at lengths about that too..... but I don't really see what relevance that has within a discussion about whether yet another POSIX kernel will "become the next Linux" because even if Kerla was to "become the next Linux" it still wouldn't satisfy your argument about diversity. So why make it in the first place?
Like I said before, I'm using a lot of good faith here assuming your misunderstanding was a genuine one and that you weren't intentionally trying to derail the conversation.
Indeed, there's a word for it which I find particularly lovely, autotelic.
I think it's the opposite. When Linux was released, basically every major tech company had their own variant of Unix. Now almost everyone has migrated to Linux or Windows.
Competing with modern Linux by creating a drop-in replacement for Linux would be a pretty difficult task given that people have been optimizing Linux for decades now, but on the other hand I think there are ways to improve dramatically on the traditional POSIX-style API that Linux mostly adheres to. For a random example, why can't a given process have more than one "current working directory" at a time? That seems like an arbitrary limitation imposed by an implementation detail in early Unix system, and it causes problems for modularity. There are many other little details like that. I think if Linux is replaced by something eventually, it'll most likely be because the new thing has a cleaner, more powerful, or more generally usable API.
(Kerla is apparently not trying to invent a new API; I'm just saying that's the direction I would recommend to anyone project with a goal of seriously competing with Linux.)
Of course that's hard to pull off in practice. Linux might have classes of errors that wouldn't exist if it were written in Rust, but even if using Rust eliminates three fourths of the code defects, the end result could be less secure if Linux gets ten or a hundred times as much scrutiny from people actively looking through the code for bugs to fix.
There wasn't many UNIXes that targeted x86 aside from 386BSD. And even those that were available were often expensive. Even Minix wasn't free at that time. Which is exactly the reason Linus created his hobby OS.
Now we have a dozen different flavours of BSD, countless Linux distributions and several OpenSolaris spin offs too. Plus many other underdogs used in speciality domains. And that's before even looking at the commercial offerings like QNX, Solaris and macOS.
Just because there are a couple of industry heavyweights that take up the majority of common use cases, don't be fooled into thinking there is a lack of choice nor even that the industry is scared to use anything outside of Linux and Windows.
> Competing with modern Linux by creating a drop-in replacement for Linux would be a pretty difficult task given that people have been optimizing Linux for decades now, but on the other hand I think there are ways to improve dramatically on the traditional POSIX-style API that Linux mostly adheres to. For a random example, why can't a given process have more than one "current working directory" at a time? That seems like an arbitrary limitation imposed by an implementation detail in early Unix system, and it causes problems for modularity. There are many other little details like that. I think if Linux is replaced by something eventually, it'll most likely be because the new thing has a cleaner, more powerful, or more generally usable API.
That's not the goal of this nor any other project that aims for ABI compatibility with Linux and there are plenty of research kernels out there if that's your thing.
> (Kerla is apparently not trying to invent a new API; I'm just saying that's the direction I would recommend to anyone project with a goal of seriously competing with Linux.)
The only way to seriously compete with Linux would be to get large commercial backing. And even then, good luck. Linux won not because it is the best but instead because it's "good enough". Same is true with Windows. Any engineer with aspirations of creating a "better kernel" or "better OS" needs to remember that.
For example, I've written a Mandelbrot visualizer so many times I've lost count. Not because the world needs another poorly written or optimized rainbow-ladybug-simulator, but because it serves as a slightly-non-trivial hello-world. For example, it's the first end-to-end thing I made in Common Lisp. https://git.sr.ht/~amtunlimited/mandelbrot-plot
I just rewrite them all the time as means to get a feeling about programming languages.
I dumped a couple of other stuff on GitHub so that HR people are happy to get a link that they never read anyway.
Then I get back to Java and .NET at the office. :)
https://www.cs.princeton.edu/~appel/modern/
It covers most areas quite alright even if a bit aged.
* Some people bake bread even though there's a good bakery nearby.
* Some people grow gardens even though there's a farmers' market down the street.
* Some people restore old cars even though there's a good restoration professional in town.
* Some people make k8s clusters on rpi even though they could rent that for cheap.
You're making an OS that's linux compatible even though there's already linux. And that is awesome!
[0]: https://gitlab.com/0xTJ/mosys/-/blob/master/src/vfs.c#L469
[1]: https://gitlab.com/0xTJ/mosys/-/blob/master/src/vfs.c#L907
If only you could write an entirely new (meant to be better, for some cases at least) kernel compatible with Linux hardware drivers - this would be not waste at all but in fact fantastic.
An idea in my mind is to use Kerla for virtualized environments like KVM where only limited device drivers are needed (virtio for example).
IMHO getting a minimal set that works is good enough to get people on board. This is still nontrivial. But for example, getting FUSE working would be useful. Even if the driver itself was not ABI compatible, it would bring functionality and someone might then aim for ABI compatibility afterward which would open even more doors.
Especially since, as I understand it, even Linux isn't compatible with the Linux driver API across versions; they can and will change internals at will and just update in-tree drivers to match. Hence some of the difficulty doing things like getting a newer kernel on assorted embedded devices (ex. 99% of phones) because you have to port the vendor's drivers to the new version and both sides changed stuff.
What do you think about other Rust OS projects like Redox?
Aside from Redox, I'd mention Tock [2], an embedded operating system. It introduces a novel components isolation using Rust's power. I believe this field is where Rust shines and am looking forward to seeing it in production.
[1]: https://www.redox-os.org/screens/ [2]: https://www.tockos.org
https://github.com/betrusted-io/xous-core
As a embedded service processor OS for a big server rack, Oxide Computer is working on 'HubrisOS'. They seem to have not released it yet, but that will be open sourced.
https://github.com/oxidecomputer
Those are two efforts where I know real resources are going into.
On Hubris and Humility: when "write your own OS" isn't the worst idea
Hubris is a small open-source operating system for deeply-embedded computer systems, such as our server's replacement for the Baseboard Management Controller. Because our BMC replacement uses a lower-complexity microcontroller with region-based memory protection instead of virtual memory, our options were limited. We were unable to find an off-the-shelf option that met our requirements around safety, security, and correctness, so we wrote one.
Hubris provides preemptive multitasking, memory isolation between separately-compiled components, the ability to isolate crashing drivers and restart them without affecting the rest of the system, and flexible inter-component messaging that eliminates the need for most syscalls -- in about 2000 lines of Rust. The Hubris debugger, Humility, allows us to walk up to a running system and inspect the interaction of all tasks, or capture a dump for offline debugging.
However, Hubris may be more interesting for what it doesn't have. There are no operations for creating or destroying tasks at runtime, no dynamic resource allocation, no driver code running in privileged mode, and no C code in the system. This removes, by construction, a lot of the attack surface normally present in similar systems.
This talk will provide an overview of Hubris's design, the structure of a Hubris application, and some highlights of things we learned along the way.
This is a great talk about what they do and why: https://www.youtube.com/watch?v=vvZA9n3e5pc
Does that mean the product will not have an FPGA? I kind of liked that idea of updating the hardware.
P.S: Twitter spaces about the history of computing are really fun. Love to hear more about all the dead computer companies you researched. There is not enough content about computer history out there.
1. ASICs are out of the question for us for a bunch of economic reasons
2. FPGAs by and large do not have a good security story
3. The ones that do (or rather, the one that does) has an entirely proprietary ecosystem and a terrible toolchain -- and seems to rely on security-through-obscurity
4. Once you are away from a softcore, the instruction set is frankly less material than the SoC -- and the RISC-V SoC space is still immature in lots of little ways relative to the Cortex space
5. Frankly, there's a lot to like about ST: good parts, good docs, good eval boards, pretty good availability (!), minimum of proprietary garbage.
We tried really hard (and indeed, I think my colleagues would say that I probably tried a little too hard) to get FPGAs to be viable, and then to get FPGAs + hardcores to be viable, and then multicore hardcores to be viable (one for the SP, one for the RoT). Ultimately, all of these paths proved to be not yet ready. And while we're big believers in FPGAs and RISC-V, we're even bigger believers in our need to ship a product! ;)
I have to say, I feel a bit “dirty” carrying so much unused and legacy code around with Linux, so I like people trying to reinvent the wheel just for the pleasure of a fresh start. For the aesthetics. Even if it’s merely a fantasy and not replacing anything soon, realistically. They are also keeping OS development accessible to new generations of geeks. The unfriendliness of C, the gigantic codebase and seemingly distinct culture make the Linux kernel quite off putting, filtering possible engagement by unfortunate parameters IMO. Novel OS development in Rust takes away at least some of those barriers and some of the gained knowledge may be applicable with the Linux kernel later.
> You can learn how printf(3) works. You can learn what happens before main().
Yes! It's such a wonderful experience. This is exactly what I most enjoy doing, just seeing how things work, maybe making my own version. I hope you have lots of fun.
You're reimplementing Linux's kernel-userspace binary interface, right? The system call interface is stable and language agnostic, it's really nice. Some pointers for anyone who'd like to know more:
https://man7.org/linux/man-pages/man2/syscalls.2.html
https://man7.org/linux/man-pages/man2/syscall.2.html
https://github.com/torvalds/linux/blob/master/Documentation/...
The permissive license would also be interesting in some applications.
IMO, writing and running tests in the kernel space is pretty easy thanks to Rust’s flexible testing feature [1].
I know this kind of stuff spark the ``it's meaningless to rewrite everything (especially Linux) in Rust'' debate. I agree 100% that rewriting Linux in Rust (or your favorite language) is just a waste of time and such a project won't live long.
That said, what I'd say here is that it's fun. Really fun. Implementing ABI compatibility requires you to understand under the hood of the kernel. You can learn how printf(3) works. You can learn what happens before main().
Especially, in Rust, it's inspiring to explore ways to write a kernel safely and expressively. One of my favorite part is the implementation of read system call [1].
Lastly, You can try this OS on SSH. Hope you enjoy :)
[1]: https://github.com/nuta/kerla/blob/main/kernel/syscalls/read...