Re:liblinux: Architecture-independent access to Linux system...
by matheusmoreira on Tuesday November 09, 2021 @10:21AM
(#29162861)
I agree. That's one thing I don't like about the kernel header. Linux system calls already have a good interface, they return error codes that consist of negated errno constants. Since we're not using the C library, we might as well do things right and remove global variables.
The kernel developers implemented nolibc.h with 3 levels of abstraction: system call entry point, function wrappers and libc interface. They set errno in the libc interface. Personally I don't see the point.
> The third level is the libc call definition.
> It exposes the lower raw sys_<name>() calls in a way that looks like what a libc usually does, takes care of specific input values, and of setting errno upon error.
Re:liblinux: Architecture-independent access to Linux system...
by remram on Tuesday November 09, 2021 @11:16AM
(#29163683)
If it's trying to be a drop-in replacement, it has to have the same API... You could also wonder why they spelled CREAT without an E again, with the same answer.
Re:liblinux: Architecture-independent access to Linux system...
by matheusmoreira on Tuesday November 09, 2021 @08:21AM
(#29161353)
Yes. My next step was implementing modern ARM architectures at least so I could compile a version for my phone. I don't have other architectures to test on so I'd need pull requests for them.
The Linux kernel nolibc.h already has support and is just better in almost every way. I wasn't sure if there was any point in continuing liblinux after I discovered it since it achieves almost everything I tried to do and doesn't even require a build system, shared libraries or anything. The support for auxiliary values is missing though. Maybe they'd accept a patch...?
Re:liblinux: Architecture-independent access to Linux system...
by Joker_vD on Tuesday November 09, 2021 @02:36AM
(#29158838)
So... it's basically kernel32.dll? That is, the shared library with a stable API across all supported platforms, and stable ABI inside any one specific platform which hides all the gritty details of switching to and back from the kernel? One wonders why it has not appeared much sooner but better late than never, I guess.
Re:liblinux: Architecture-independent access to Linux system...
by guerrilla on Tuesday November 09, 2021 @02:41AM
(#29158874)
The explanation for why it's useful is at the bottom of the page, but it wasn't necessary before glibc got behind and before Linux started adding things that glibc would probably never implement*. Generally, on UNIX-like systems, the C library isn't just the C library but also exposes the UNIX/POSIX API which is just a set of wrappers around calling into the kernel (done with software interrupt 0x80 on x86 Linux.)
* Remember that glibc is part of the GNU project and is meant to be portable beyond Linux, especially to Hurd. It might not make sense for them to implement interfaces that wouldn't work or would require significant work to emulate on Hurd, the BSDs and Solaris.
Re:liblinux: Architecture-independent access to Linux system...
by Joker_vD on Tuesday November 09, 2021 @02:52AM
(#29158919)
Well, yeah, merging kernel calls into the runtime of one specifically favoured programming language is a silly idea, why not have the kernel calls in a separate, cleanly defined language-agnostic OS-specific library? "It just the way the things has come to work historically"?
Re:liblinux: Architecture-independent access to Linux system...
by guerrilla on Tuesday November 09, 2021 @03:03AM
(#29158968)
Well, first of all the UNIX/POSIX API is a C API, so that question doesn't make much sense. On top of that, there have been no competitive systems programming languages until recently (and certainly none that weren't written in C), so that wouldn't have been worth it. Also, it wasn't really an idea per se: C is just the native language and lingua franca of UNIX and that's just how UNIX developed organically.
Re:liblinux: Architecture-independent access to Linux system...
by Joker_vD on Tuesday November 09, 2021 @03:13AM
(#29159017)
I would argue that your post is a nice post-rationalization, and is not very accurate historically. The mans in the first versions of UNIX used to show how to do syscalls both in Fortran and C, and only later they've dropped the Fortran because it was too much hassle; and since it was all done way before Ulrich Drepper invented shared libraries, we ended with system calls buried in the C runtime. Also, people wrote complete working OSes in Pascal exactly around the time C was invented, so the part about "no competing systems programming languages" is not true either.
Re:liblinux: Architecture-independent access to Linux system...
by Joker_vD on Tuesday November 09, 2021 @03:59AM
(#29159278)
Sorry, I was wrong; for some reason I thought it was Fortran but of course it was PDP-11 assembly. It stayed there up until v7, and was removed in v8. That subtracts from my argument.
Re:liblinux: Architecture-independent access to Linux system...
by hnlmorg on Tuesday November 09, 2021 @03:09AM
(#29159004)
> On top of that, there have been no competing systems programming languages until very recently
There have been plenty. What has changed isn't the lack of competing languages, it's the discontent with C and C++. The Rust hype has coincided with the anti-C and anti-C++ hype and thus the "rewrite it in Rust" movement was born. But Rust wasn't even the first memory safe systems programming language, let alone first alternative to C (and neither was C the first systems programming language itself).
Re:liblinux: Architecture-independent access to Linux system...
by guerrilla on Tuesday November 09, 2021 @03:44AM
(#29159181)
I meant competitive and edited my post now to reflect that and I stand by it. I can only speak for myself with certainty but the benefits never outweighed the costs until more recent languages. If there were really alternatives then more things would have been written in them instead of in C or moving to higher-level languages.
Re:liblinux: Architecture-independent access to Linux system...
by hnlmorg on Tuesday November 09, 2021 @04:00AM
(#29159284)
> I meant competitive and edited my post now to reflect that and I stand by it. I can only speak for myself with certainty but the benefits never outweighed the costs until more recent languages but if there were really alternatives then more things would have been written in them instead of in C or moving to higher-level languages.
You're now conflating "popular" with "better". What makes a language successful isn't always the same as what makes it better (eg there are countless examples of languages failing to gain significant traction (like Zig, Nim, OCaml, D, ) over other languages that had a large company backing it (like Rust did).
Moreover, what makes a language "better" isn't going to be the same for everyone. Some crave simplicity so that the entire language can be memorised (like SPARK) whereas others crave complexity where they can chose the subset of features to support (like C++). Some like C-style braces, some like ALGOL style keywords (like Ada), some prefer S-Expressions (like LISP) and some people even prefer to go direct to assembly.
We've seen operating systems built in Fortran, Pascal, LISP, Oberon and others so it's not even as if I'm making a theoretical point here.
Re:liblinux: Architecture-independent access to Linux system...
by guerrilla on Tuesday November 09, 2021 @04:14AM
(#29159372)
> What makes a language successful isn't always the same as what makes it better.
That's definitely true but I think the odds are that at least one of them would have succeeded if many of them really were competitive. They got a lot of chances.
Re:liblinux: Architecture-independent access to Linux system...
by hnlmorg on Tuesday November 09, 2021 @04:26AM
(#29159435)
You've completely missed the point.
Let's put it in other terms: if your point were true then Javascript wouldn't be the language of browsers. English (which is a mess of a language) wouldn't be the de facto international language.
You cannot conflate "success" with "better". There are so many other factors at play (like money, company politics and which platforms that language is used on happened to luck their way into the mainstream...to name a few).
My point being that the reasons for C's success was as much to do with external factors as it was to do with C itself. In fact in a parallel universe where Bell Labs didn't give UNIX source away and RMS wasn't born (thus wasn't around to create GNU and mandate C as the standard), I'd wager you'd be talking about an ALGOL-derived language (possibly SPARK) as "the best" with C as an academic curiosity; a footnote in history. Or maybe we'd all be writing LISP.
Re:liblinux: Architecture-independent access to Linux system...
by guerrilla on Tuesday November 09, 2021 @04:47AM
(#29159577)
Please stop making accusations. It's obvious that being better is not a sufficient condition for being popular. It's obvious that C and JavaScript are popular now primarily due to entrenchment. I'm not saying that there were no "better" languages by the standards you mean. What I'm saying is that none of them were better enough. They needed to amass such a large number of pros so as to outweigh that one single con of not being the entrenched language for the niche that we're speaking about.
This goes for ISAs too: We use x86 not because it's better or even good. It's because PowerPC (or whatever you prefer) and all those that were better were not better enough to outweigh the massive cost of losing backward compatability, training, habits, relevant capital and all of that. Maybe RISC V, paired with the right microarchitecture, actually does turn out to be so much better that the transition becomes worth that huge cost. We'll see.
We will overcome C, JavaScript, x86 and all that but only once the contenders to displace them are just *that* much better. Many things have had the chance to do that and were better in many ways but not until recently has the balanced tipped, in my opinion.
Re:liblinux: Architecture-independent access to Linux system...
by hnlmorg on Tuesday November 09, 2021 @05:12AM
(#29159744)
> Please stop making accusations. It's obvious that being better is not a sufficient condition for being popular.
I'm not making accusations. You literally said:
> That's definitely true but I think the odds are that at least one of them would have succeeded if many of them really were competitive. They got a lot of chances.
Which is the opposite of the point you're now making. I can only respond to the message you post.
Any way, I think we are largely in agreement even if we're describing the problem from different angles.
Re:liblinux: Architecture-independent access to Linux system...
by guerrilla on Tuesday November 09, 2021 @06:32AM
(#29160278)
When you say I'm missing the point, confounding things or making the opposite point, you're making accusations.
Both of my points are congruent. Rephrase the quote with "competitive" replaced with "better enough" and this becomes obvious. Your original points were that competitive (i.e. better enough) languages existed and people just got sick of C. I disagreed saying that no, if there had been competitive (i.e. better enough) languages then at least one of them would have succeed by now as they'd all had plenty of chances.
Another argument against this "sick of C" idea is that if that were really true then any of the old languages could have been adopted, now that people are supposedly suddenly sick of C. But that's not what's happening. The new languages are being adopted because they're finally better enough. The old languages aren't being adopted because they were never competitive.
Re:liblinux: Architecture-independent access to Linux system...
by GoblinSlayer on Tuesday November 09, 2021 @08:27AM
(#29161410)
In the early days of microcomputers the competition was between basic and pascal. Basic was interpreted because decent compilers on that hardware were difficult to achieve, but it was achieved for pascal anyway, so it won. C somehow came later.
Re:liblinux: Architecture-independent access to Linux system...
by kazinator on Tuesday November 09, 2021 @07:37PM
(#29169582)
Part of that "somehow" may have been Microsoft. They provided a C development environment for MS-DOS, and developed Windows in C, imitating Unix. That probably came in via Xenix. (MS-DOS originally had "Xenix" system calls and features aped after Unix, which tells you that the developers were familiar with Xenix and thus had contact with C and Unix.) Windows API's being expressed in C might have been the catalyst that caused C to explode in the consumer PC scene.
People in the computing field would have known about C in connection with Unix, but Microsoft brought system software written in C to the mass consumer market, smack in the 1980's. That would have raised the profile of C as something proven that could be used to deliver; and you could use the compiler from that same vendor, if you wanted.
Re:liblinux: Architecture-independent access to Linux system...
by hnlmorg on Tuesday November 09, 2021 @05:53PM
(#29168652)
Sorry but you're largely wrong on all counts there.
1. There was a lot more around than just BASIC and Pascal,
2. Pascal wasn't born much before C (roughly a couple of years),
3. and BASIC compilers did exist even as early as in the 60s.
Some examples of other languages (all of which pre-date Pascal and C):
Pascal itself was inspired from ALGOL. So there's a whole class of languages you're excluding. ALGOL was typically compiled too.
Also compiled was FORTRAN. In fact the first FORTRAN compiler was written in the 1957 and the language is credited as being the first ever high level programming language.
Even BASIC wasn't one of a kind. There were a few languages similar (the names of which escape me at this point in time). But BASIC was definitely the most popular of it's ilk.
And lets not forget LISP. It's the second oldest high level language after FORTRAN. LISP has had compilers around since the 60s, although the language is older than that.
Another popular language with compilers back in the 60s was COBOL. And as you maybe aware, some banks are still stuck running COBOL code (albeit modern COBOL looks nothing like 60s COBOL)
And I'm just listing the highlights here. I could also talk about how C evolved from B and B evolved from BCPL. But BCPL's syntax is a lot more like ALGOL than it is C. I could also talk about how FORTRAN, ALGOL and COBOL themselves aren't really a single language but rather a class of languages too. In fact Pascal evolved from ALGOL W and ALGOL W compiler was written in PL/360 which is another ALGOL-like systems language designed specifically for the IBM System/360. Which leads nicely onto the next point that there was also a lot of machine specific or vendor specific languages out there too. In fact it was quite common for organisations to design their own language - possibly even more so then than it is now.
So no, it wasn't just a decision between BASIC and Pascal in the early days. There were a lot of choices even back then. And in fact Pascal wasn't even an option in the early days because it didn't come onto the scene until 1970. Verses the 50s and 60s for all the other languages mentioned above.
Re:liblinux: Architecture-independent access to Linux system...
by kazinator on Tuesday November 09, 2021 @07:28PM
(#29169510)
It seems you might have paid insufficient attention to the "in the early days of microcomputers ..." qualifying remark in the grandparent comment.
When microcomputers emerged in the 1970's, in particular consumer microcomputers, many languages and other software simply did not migrate to them from Big Iron, mainly for reasons that it just didn't fit.
People who started computing with microcomputers in that era would only have read about Algol or Cobol in Byte or Creative Computing; it wouldn't be found running on their Comodore Pet or what have you.
Re:liblinux: Architecture-independent access to Linux system...
by hnlmorg on Wednesday November 10, 2021 @03:06AM
(#29172254)
Ack yes, I did miss the "micro" part of this sentence. Sorry.
That all said, you'd still be surprised at the number of programming languages that were ported to microcomputers, albeit more so in the early to mid-80s rather than the very start of their evolution in the 70s.
I remember seeing a similar array of options for other microcomputers too.
But I'm not disagreeing with the GPs point, those solutions I've exampled were pretty niche. I raise it more for academic interest than as a counterargument.
Re:liblinux: Architecture-independent access to Linux system...
by koprulusector on Tuesday November 09, 2021 @05:35AM
(#29159897)
The kernel is written in C. Ok, why does that matter? Because its interfaces and architecture are tightly intertwined with C. Remember that C was partially developed to use in building Unix.
So, kernel was written in C and therefore uses C data structures, pointers, and follows C conventions for using them (for example, kernel might return a struct of pointers). This applies to interfaces and structures, making syscalls (depending on which syscall and what is needed), checking return values or error codes.
You don’t NEED this stuff exactly to run on Unix. What and how much you need, or even just convenience, are how it boils down. If you want to use eBPF to trace things from raw network packets, syscalls, file systems, you can do some in Python but you’ll need some degree of C.
You want to run a Django or flask app in just Python? Good to go.
Re:liblinux: Architecture-independent access to Linux system...
by johannes1234321 on Tuesday November 09, 2021 @08:05AM
(#29161185)
Unix can run other languages.
But for answering your question deeper, we have to find an understanding what boundary we're talking about. The kernel syscall interface? Things like Linux's vdso? POSIX library (libc)?
The more we go on in that sequence the more C we get into the interfaces and many of those APIs are essential for writing well-behaving non-trivial programs.
The closer we stay to the kernel's syscall interface the less C you get, but still the syscall interface and C are related. Strings are often NULL-terminated. Arguemtns are passed (via registers) from left to right, not like in Pascal convention right to left. (First arg always in EBX, second in EXC etc.)
A different language will need some interoperability with C conventions for being a good citizen on Unix.
A Pascal operating system for instance would use Pascal strings (historically what tis now called ShortString) where length is encoded in the first byte.
Re:liblinux: Architecture-independent access to Linux system...
by Joker_vD on Tuesday November 09, 2021 @08:52AM
(#29161698)
> A Pascal operating system for instance would use Pascal strings
Only if the designers would make that decision, and they are not in any way obliged to make such decision! Nobody stops them from using e.g. a moral equivalent of "RECORD size: CARDINAL; str_ptr: ^CHAR END" for passing strings in their system calls (which would probably coincide with using "size_t size, char *str_ptr" in the C prototype at the binary level), or something else entirely.
Is the idea of an OS not exposing its implementation language in its public API that foreign so I have to express it three times in this thread and every time be replied with "No, what are you talking about, of course the OS would use reuse the structures of the runtime of the language that it is written in in its public syscall API"?
Gee, I wonder how the syscall calling convention would look if I wrote an OS entirely in assembly.
Re:liblinux: Architecture-independent access to Linux system...
by johannes1234321 on Tuesday November 09, 2021 @09:13AM
(#29161961)
It's not what the implementation uses, but "which conventions does the call interface assume" you can use Pascal to create an operating system with a system interface that benefits Fortran. (I know too little about Fortran to describe what that would look like)
With C and Unix the issue of course is that both were developed around the same time by the same people for working together.
Re:liblinux: Architecture-independent access to Linux system...
by GoblinSlayer on Tuesday November 09, 2021 @09:27AM
(#29162141)
Or it could be designed for interoperability like ODBC which was designed to be compatible with languages that have ISO standards: Ada, Fortran, C, COBOL, Pascal without tightly locking to either.
And yes, with common Pascal compilers i can write specific functions in a way to provide a C calling interface (called STDCALL) so those functions can be called from C. But for reaching arbitrary Pascal functions I need my own assembly code which passes parameters in the right order and manages the stack "correctly" (in C the caller clears the stack, in Pascal the calleé)
Re:liblinux: Architecture-independent access to Linux system...
by Joker_vD on Tuesday November 09, 2021 @10:13PM
(#29170645)
...the calling conventions are not the properties of the language itself, or the OS, only of the langauge implementations — the Gnu Pascal Compiler uses the same calling convention as the Gnu C Complier [0], so I think GCC simply decided not to support any other Pascal implementation explicitly and left those to figure out themselves how to link with object files produced by GCC. And yes, generally different implementations of the same language tended to use the same calling conventions to be compatible/interoperable with each other in the first place, although it's not a universal rule, so that's how why x86 calling conventions were labeled with PL names; but that's slight misnomering.
On x64 Linux, everyone managed to agree to use the same calling convention which is of course very convenient. The language of it's written specification is heavily C-biased, but if you squint you can see that it talks mostly about eightbytes and and compound objects and where to shove those so it more or less trivially translates to most other Algol- or C-descendant imperative languages; I believe it can even be reformulated strictly in language-neutral/assembly terms.
On vanilla x86, of course, everyone invented their own calling convention, even different implementations of the same languages: IIRC, Borland C++, MSVC, Watcom, and GCC on Windows all by default produced object files that you could not simply link together without manual tinkering. And (32-bit) Windows, despite being written in C, uses a variation of "Pascal" calling convention in its system API which they called "stdcall". I believe there were some C compilers that actually used "stdcall" by default instead of "cdecl".
Hell, even on x64, Cygwin GCC (on Windows) uses SysV ABI for everything except interfacing with Windows itself where it obviously has to use Windows x64 calling convention.
Re:liblinux: Architecture-independent access to Linux system...
by hnlmorg on Tuesday November 09, 2021 @03:00AM
(#29158958)
Because every programming language will have their own opinions on how to link to external libraries and what those external libraries should look like (eg Java libraries are ZIP archives with a .jar extension, JIT languages will have their libraries as raw source code).
But since C is the de facto standard for OS libraries, most languages end up supporting calls to C libraries. Thus C libraries do become the language-agnostic library.
Re:liblinux: Architecture-independent access to Linux system...
by matheusmoreira on Tuesday November 09, 2021 @11:14AM
(#29163661)
Higher level languages such as C++ have extremely complex binary interfaces. Only C++ code dares to link against C++ code. Even then there are problems between different compilers and even different versions of the same compiler. New standard version comes out, it requires that standard classes have certain performance characteristics... Suddenly compilers are breaking binary compatibility in order to restablish conformance.
So in order to have usable libraries people will wrap their C++ implementation in C interfaces. Rust faces the same problem. All those nice abstractions result in a complex ABI that will never touched by anything outside the Rust ecosystem. There is no escaping the extern "C".
Some languages are virtualized. Java, C#, Ruby, Python, Javascript all have virtual machines. There's no way to interface directly with code running in those languages. We must interface with the virtual machine instead: import it into our process and ask it to call the functions we want to use.
C binary interfaces are not language agnostic but they are quite simple. The result of compiling a C library is symbols pointing to functions with simple calling conventions. As a result, virtually any language can look up these symbols and call them.
Re:liblinux: Architecture-independent access to Linux system...
by matheusmoreira on Tuesday November 09, 2021 @02:05PM
(#29165960)
Yes. The Linux system call calling convention doesn't match the C ones at all. One needs inline assembly in order to interface with the Linux kernel from C.
The ideal solution is to have a system_call keyword built into the compiler that emits the proper instructions for the target platform.
Re:liblinux: Architecture-independent access to Linux system...
by couchand on Tuesday November 09, 2021 @08:06AM
(#29161206)
That doesn't ring true to me. My FFI experience is somewhat limited (JS to C, JS to C++ via C wrappers, Rust to C, Rust to C++ via C wrappers) but from what I've seen you have to speak C to make FFI.
Re:liblinux: Architecture-independent access to Linux system...
by GoblinSlayer on Tuesday November 09, 2021 @10:00AM
(#29162594)
Few languages truly speak full C (e.g. through clang plugin), they support ABI, but there are libraries like libpng and gtk with API disconnected from ABI, for them FFI doesn't resemble heavily macro encumbered C API.
On Windows and other operating systems, things such as system call numbers can and do change. There's no way to interface with the kernel without linking to vendor libraries.
So on Linux you can ditch libc and write your own code. A Rust liblinux is definitely possible, I tested it. You can even have a compiler generate the system calls for you! It just needs to support the simple calling convention. No libraries necessary. I've seen a project that does just that:
Re:liblinux: Architecture-independent access to Linux system...
by matheusmoreira on Tuesday November 09, 2021 @01:25PM
(#29165338)
Amazing!! Rust safety all the way up to the Linux system calls, and no legacy cruft like errno anywhere. That's the sort of project I had in mind. I'm so glad I'm not insane for imagining things like this. It would be so great to see a 100% Rust or Lisp or Zig user space built on top of Linux. Would be a lot more interesting than the same old POSIX user space every distribution ships with...
Re:liblinux: Architecture-independent access to Linux system...
by Joker_vD on Tuesday November 09, 2021 @09:16AM
(#29161988)
I mean, I also can call functions from kernel32.dll on Windows without need of any libc, and the ABI is stable and will not change; and if/when MS decides to change the actual mechanism of switching into the kernel mode, or decide to move some stuff from the kernel into the user space ― they can, and I don't have to worry about that (and recompile my apps) because it's all abstracted away! "Fix it once in the shared library, and all its clients will benefit automatically".
On Linux, however, we have, for example, glibc on one hand which claims that it doesn't have a goal of forwarding all of the Linux's syscall, and several failed initiatives on the other hand to allow the user's code to perform syscalls only from inside of glibc (which spells doom to anything that doesn't use libc, e.g. Golang) because of security reasons. And the history of still having to support INT 80h together with SYSCALL on x64 on the third hand.
Re:liblinux: Architecture-independent access to Linux system...
by matheusmoreira on Tuesday November 09, 2021 @10:40AM
(#29163152)
> several failed initiatives on the other hand to allow the user's code to perform syscalls only from inside of glibc (which spells doom to anything that doesn't use libc, e.g. Golang) because of security reasons
I didn't know about these efforts. Please elaborate.
The basic idea is to make it harder for remote code execution bugs to result in useful abilities. Restricting access to syscalls to libc, and using adress randomization to make it hard to find libc makes it harder to do system calls when you take over a program.
Re:liblinux: Architecture-independent access to Linux system...
by matheusmoreira on Tuesday November 09, 2021 @12:31PM
(#29164599)
I see. That's really interesting. Looks like it is implemented by mprotecting the pages where the trusted code resides. The program's text segments are considered trusted by default so a freestanding C program that makes system calls directly should not be affected.
> For static binaries, the valid regions are the base program's text segment and the signal trampoline page.
I'm not sure but it looks like the BSD libc is some kind a special case? Surely this is not necessary? A compiler or language runtime should be able to somehow mark those segments with the appropriate permissions.
Re:liblinux: Architecture-independent access to Linux system...
by Joker_vD on Tuesday November 09, 2021 @11:23AM
(#29163763)
OpenBSD does it already [0], see also discussions here on HN at [1]. Thankfully, the push to implement something like that in Linux wasn't strong enough. By the way, notice how questions "why do I have to go through an OS-approved C runtime even if I write in some other language?" are responded with "Oh come on, in the end, everything is built on top of libc".
The problem, again, is not that this kernel-entry interface is factored out into a separate shared library; the problem is that it's glued into a runtime of one specific programming language that now apparently everybody has to use.
Re:liblinux: Architecture-independent access to Linux system...
by matheusmoreira on Tuesday November 09, 2021 @12:55PM
(#29164943)
> notice how questions "why do I have to go through an OS-approved C runtime even if I write in some other language?" are responded with "Oh come on, in the end, everything is built on top of libc"
Yeah. I don't like that. Just because that's the way it works now doesn't mean it must work that way forever. Because of its stable system calls, Linux doesn't impose any design on user space. People could theoretically rebuild the entire system in Lisp or Rust or something. Potential is limitless. Also, it's amazing that I can compile a freestanding program with zero dependencies and boot Linux straight into it.
> the problem is that it's glued into a runtime of one specific programming language that now apparently everybody has to use
I agree. It's a really hard problem to solve. Since it's based on the POSIX APIs there are tons of definitions and data types that are specified exclusively in C and its preprocessor. Not exactly accessible for other languages. The preprocessor macros in particular are extremely problematic. The only sane way to untangle those is to use a C compiler.
There's an interesting discussion in that thread you linked. What if the vDSO was the entry point for all system calls? That would certainly be interesting.
I think the vDSO ABI is underspecified. It's just the C ABI, whatever that means on your system. This can cause bugs due to assumptions. For example, this Go bug happened because Go called hardened versions of the vDSO functions with insufficient stack space:
Re:liblinux: Architecture-independent access to Linux system...
by phendrenad2 on Tuesday November 09, 2021 @02:26AM
(#29158781)
Very cool. It seems that glibc is not a faithful wrapper of the Linux syscalls (it makes assumptions about some calls, to provide a better C API, and doesn't support some Linux-specific syscalls).
Re:liblinux: Architecture-independent access to Linux system...
by GoblinSlayer on Tuesday November 09, 2021 @03:37AM
(#29159138)
>The C library is itself a user space program, so the rule that a user space program has to go through the C library obviously isn't logical. And the GNU C library is an optional tool, not an official part of the kernel interface (if it were the latter, I would expect to see it packaged with the kernel).
Ohh, if only we could make C the second class citizen. C brings with itself meaningless legacy from 1969 into modern systems, like you might think openat2 was created recently, but it's designed for PDP-7, because it's C and stuff was always done this way.
Re:liblinux: Architecture-independent access to Linux system...
by matheusmoreira on Tuesday November 09, 2021 @08:47AM
(#29161634)
With Linux, you can! I started this project years ago just to prove to myself that it was possible.
Every Linux process interfaces through the kernel via the system call interface. It is stable and language agnostic. The interface is defined at the processor architecture level.
Any compiler can emit this sort of code, including JIT compilers! It's possible to discard the entire C user space and rewrite everything in Rust. Would be a monumental effort but still.
Note that no matter what there will be at least one C library remaining: the vDSO the kernel maps into the address space of every process.
Apparently for BSD, the ABI is not that stable, glibc works around bugs on some platforms (adding memory barriers etc), and there are security features that check the origin of syscalls. I wonder whether this is truly acceptable on Linux...
Re:liblinux: Architecture-independent access to Linux system...
by matheusmoreira on Tuesday November 09, 2021 @01:30PM
(#29165413)
It is my understanding that Go attempted to perform direct system calls on systems where it was never supported. As a result, Go programs stopped working when the binary interface changed.
This is not a problem on Linux.
> glibc works around bugs on some platforms (adding memory barriers etc)
That's really interesting. Do you have examples of such fixes? I'm more familiar with musl sources, glibc proved to be somewhat difficult to navigate.
Re:liblinux: Architecture-independent access to Linux system...
by lelanthran on Wednesday November 10, 2021 @01:44AM
(#29171807)
> Any compiler can emit this sort of code, including JIT compilers! It's possible to discard the entire C user space and rewrite everything in Rust.
Well, yeah, but that's a tautology, no?
A willingness to discard everything indicates a willingness to rewrite everything. So, yeah, it is possible to rewrite everything if you are willing to rewrite everything.
Re:liblinux: Architecture-independent access to Linux system...
by matheusmoreira on Wednesday November 10, 2021 @09:51AM
(#29175187)
A willingness to discard everything doesn't mean you can actually do that though. Windows, for example, simply doesn't let you get rid of the Microsoft DLLs. You can't talk to the kernel directly, the interface will change and break your program. The BSDs and macOS all do the same thing.
Linux lets you do this. The interface is stable. People can build their custom user space on top of it and enjoy a great kernel full of drivers. Wouldn't it be amazing if someone came up with a 100% Lisp user space? A 100% Rust user space?
Re:liblinux: Architecture-independent access to Linux system...
by lelanthran on Thursday November 11, 2021 @05:37AM
(#29186529)
> Linux lets you do this. The interface is stable. People can build their custom user space on top of it and enjoy a great kernel full of drivers. Wouldn't it be amazing if someone came up with a 100% Lisp user space? A 100% Rust user space?
Great indeed (I like Lisp!), but I have to question how useful that would be in practice.
Let's take Lisp as an example: I can now program only in Lisp and talk to the OS in native Lisp only (including POSIX calls). The downsides are that I cannot call any existing library; my 100% Lisp environment cannot use libzip, or libffmpeg/libavutil, or libSDL, etc.
In order to make that 100% Lisp environment actually useful it's going to be doing FFI calls anyway into a C ABI. That makes the OS-interface redundant.
Python is as popular as it is purely because it made calling into the C ABI easy: existing libraries were made instantly useful in Python, not unnecessarily hard.
Re:liblinux: Architecture-independent access to Linux system...
by GoblinSlayer on Tuesday November 09, 2021 @05:46AM
(#29159975)
More precisely it's about making PDP-7 design the second class citizen. Interoperable design becomes the first class instead. POSIX would be implemented on top of that.
Re:liblinux: Architecture-independent access to Linux system...
by lelanthran on Tuesday November 09, 2021 @06:24AM
(#29160229)
I understand that, but right now the C ABI is first class, and every language can call into it.
What replacement are you proposing that every language can call into? How would a migration of existing Java, Python, Ruby, C++ etc programs be done to this new interface? Do programs need to be modified at the source level? Just recompiled? How will the existing programs that cannot be recompiled be handled?
I quite aware that you can make the C ABI a shim into the new first class ABI to make existing programs work, but why would anyone target the new ABI when the existing one still works?
Re:liblinux: Architecture-independent access to Linux system...
by lelanthran on Tuesday November 09, 2021 @01:23PM
(#29165306)
> Obviously it's not aimed at people who want to use only POSIX.
Okay, then it's aimed at people who want to use more than just POSIX, right? That's the entire world.
In which case the question still applies - how do you migrate everyone over to your new interoperable ABI? How will I run Gimp, Firefox/Thunderbird, IntelliJ, VSCode, Slack, Xterm, all my command-line tools (tar, grep, etc), Git, Meld, Starcraft 2 (and all 100+ games in my Steam library, and Steam itself) and more?
How do you see your replacement running MySQL, Apache, Nginx, PostgreSQL, bash/zsh/etc, SQLite, stunnel, netcat, wireshark/tcpdump, etc?
Do they all have to be recompiled? Replaced with new sootware?
Because each and every one of the software I listed above depends on a C ABI. And any shim provided to make existing software work will simply result in developers ignoring your new ABI and targeting the C ABI for maximum compatibility.
Re:liblinux: Architecture-independent access to Linux system...
by matheusmoreira on Tuesday November 09, 2021 @10:47AM
(#29163269)
Indeed, the library is missing wrappers for most of the system calls. I wrote the functions I needed for the code in the examples directory. I even wrote a little script that lets me see which system calls haven't been implemented yet.
make system-calls.missing
The library does provide access to every system call through the generic system_call function. It just doesn't have compile time type checking.
The part most relevant to this discussion:
> I stopped developing this because I discovered the Linux itself has a better solution that they use for their own tools:
> https://github.com/torvalds/linux/blob/master/tools/include/...
1: https://news.ycombinator.com/item?id=29106179
The kernel developers implemented nolibc.h with 3 levels of abstraction: system call entry point, function wrappers and libc interface. They set errno in the libc interface. Personally I don't see the point.
> The third level is the libc call definition.
> It exposes the lower raw sys_<name>() calls in a way that looks like what a libc usually does, takes care of specific input values, and of setting errno upon error.
Defining NOLIBC_IGNORE_ERRNO solves the problem!
> Currently, the only supported architecture and compiler is x86_64 and GCC, respectively
Let's wait until its architecture-independence has arrived in practice.
The Linux kernel nolibc.h already has support and is just better in almost every way. I wasn't sure if there was any point in continuing liblinux after I discovered it since it achieves almost everything I tried to do and doesn't even require a build system, shared libraries or anything. The support for auxiliary values is missing though. Maybe they'd accept a patch...?
I also asked Greg on reddit about nolibc.h once:
https://old.reddit.com/r/linux/comments/fx5e4v/im_greg_kroah...
Turns out there was a klibc project that never turned up in my original research!
* Remember that glibc is part of the GNU project and is meant to be portable beyond Linux, especially to Hurd. It might not make sense for them to implement interfaces that wouldn't work or would require significant work to emulate on Hurd, the BSDs and Solaris.
I don't know what early versions and manuals you mean because that wasn't in v6 or v7. Was there Fortran on UNIX before F77 was introduced in v7?
There have been plenty. What has changed isn't the lack of competing languages, it's the discontent with C and C++. The Rust hype has coincided with the anti-C and anti-C++ hype and thus the "rewrite it in Rust" movement was born. But Rust wasn't even the first memory safe systems programming language, let alone first alternative to C (and neither was C the first systems programming language itself).
You're now conflating "popular" with "better". What makes a language successful isn't always the same as what makes it better (eg there are countless examples of languages failing to gain significant traction (like Zig, Nim, OCaml, D, ) over other languages that had a large company backing it (like Rust did).
Moreover, what makes a language "better" isn't going to be the same for everyone. Some crave simplicity so that the entire language can be memorised (like SPARK) whereas others crave complexity where they can chose the subset of features to support (like C++). Some like C-style braces, some like ALGOL style keywords (like Ada), some prefer S-Expressions (like LISP) and some people even prefer to go direct to assembly.
We've seen operating systems built in Fortran, Pascal, LISP, Oberon and others so it's not even as if I'm making a theoretical point here.
That's definitely true but I think the odds are that at least one of them would have succeeded if many of them really were competitive. They got a lot of chances.
Let's put it in other terms: if your point were true then Javascript wouldn't be the language of browsers. English (which is a mess of a language) wouldn't be the de facto international language.
You cannot conflate "success" with "better". There are so many other factors at play (like money, company politics and which platforms that language is used on happened to luck their way into the mainstream...to name a few).
My point being that the reasons for C's success was as much to do with external factors as it was to do with C itself. In fact in a parallel universe where Bell Labs didn't give UNIX source away and RMS wasn't born (thus wasn't around to create GNU and mandate C as the standard), I'd wager you'd be talking about an ALGOL-derived language (possibly SPARK) as "the best" with C as an academic curiosity; a footnote in history. Or maybe we'd all be writing LISP.
This goes for ISAs too: We use x86 not because it's better or even good. It's because PowerPC (or whatever you prefer) and all those that were better were not better enough to outweigh the massive cost of losing backward compatability, training, habits, relevant capital and all of that. Maybe RISC V, paired with the right microarchitecture, actually does turn out to be so much better that the transition becomes worth that huge cost. We'll see.
We will overcome C, JavaScript, x86 and all that but only once the contenders to displace them are just *that* much better. Many things have had the chance to do that and were better in many ways but not until recently has the balanced tipped, in my opinion.
I'm not making accusations. You literally said:
> That's definitely true but I think the odds are that at least one of them would have succeeded if many of them really were competitive. They got a lot of chances.
Which is the opposite of the point you're now making. I can only respond to the message you post.
Any way, I think we are largely in agreement even if we're describing the problem from different angles.
Both of my points are congruent. Rephrase the quote with "competitive" replaced with "better enough" and this becomes obvious. Your original points were that competitive (i.e. better enough) languages existed and people just got sick of C. I disagreed saying that no, if there had been competitive (i.e. better enough) languages then at least one of them would have succeed by now as they'd all had plenty of chances.
Another argument against this "sick of C" idea is that if that were really true then any of the old languages could have been adopted, now that people are supposedly suddenly sick of C. But that's not what's happening. The new languages are being adopted because they're finally better enough. The old languages aren't being adopted because they were never competitive.
People in the computing field would have known about C in connection with Unix, but Microsoft brought system software written in C to the mass consumer market, smack in the 1980's. That would have raised the profile of C as something proven that could be used to deliver; and you could use the compiler from that same vendor, if you wanted.
1. There was a lot more around than just BASIC and Pascal,
2. Pascal wasn't born much before C (roughly a couple of years),
3. and BASIC compilers did exist even as early as in the 60s.
Some examples of other languages (all of which pre-date Pascal and C):
Pascal itself was inspired from ALGOL. So there's a whole class of languages you're excluding. ALGOL was typically compiled too.
Also compiled was FORTRAN. In fact the first FORTRAN compiler was written in the 1957 and the language is credited as being the first ever high level programming language.
Even BASIC wasn't one of a kind. There were a few languages similar (the names of which escape me at this point in time). But BASIC was definitely the most popular of it's ilk.
And lets not forget LISP. It's the second oldest high level language after FORTRAN. LISP has had compilers around since the 60s, although the language is older than that.
Another popular language with compilers back in the 60s was COBOL. And as you maybe aware, some banks are still stuck running COBOL code (albeit modern COBOL looks nothing like 60s COBOL)
And I'm just listing the highlights here. I could also talk about how C evolved from B and B evolved from BCPL. But BCPL's syntax is a lot more like ALGOL than it is C. I could also talk about how FORTRAN, ALGOL and COBOL themselves aren't really a single language but rather a class of languages too. In fact Pascal evolved from ALGOL W and ALGOL W compiler was written in PL/360 which is another ALGOL-like systems language designed specifically for the IBM System/360. Which leads nicely onto the next point that there was also a lot of machine specific or vendor specific languages out there too. In fact it was quite common for organisations to design their own language - possibly even more so then than it is now.
So no, it wasn't just a decision between BASIC and Pascal in the early days. There were a lot of choices even back then. And in fact Pascal wasn't even an option in the early days because it didn't come onto the scene until 1970. Verses the 50s and 60s for all the other languages mentioned above.
When microcomputers emerged in the 1970's, in particular consumer microcomputers, many languages and other software simply did not migrate to them from Big Iron, mainly for reasons that it just didn't fit.
People who started computing with microcomputers in that era would only have read about Algol or Cobol in Byte or Creative Computing; it wouldn't be found running on their Comodore Pet or what have you.
That all said, you'd still be surprised at the number of programming languages that were ported to microcomputers, albeit more so in the early to mid-80s rather than the very start of their evolution in the 70s.
Take the British Amstrad CPC 464, that had compilers for Forth, C and COBOL (to name but a few): https://www.cpcwiki.eu/index.php/Category:Programming_softwa...
I remember seeing a similar array of options for other microcomputers too.
But I'm not disagreeing with the GPs point, those solutions I've exampled were pretty niche. I raise it more for academic interest than as a counterargument.
So, kernel was written in C and therefore uses C data structures, pointers, and follows C conventions for using them (for example, kernel might return a struct of pointers). This applies to interfaces and structures, making syscalls (depending on which syscall and what is needed), checking return values or error codes.
You don’t NEED this stuff exactly to run on Unix. What and how much you need, or even just convenience, are how it boils down. If you want to use eBPF to trace things from raw network packets, syscalls, file systems, you can do some in Python but you’ll need some degree of C.
You want to run a Django or flask app in just Python? Good to go.
[This article on Linux ABI](https://en.wikipedia.org/wiki/Application_binary_interface) might help explain better/in more detail.
Sorry if this isn’t the best explanation, I’m on mobile and up too late.
But for answering your question deeper, we have to find an understanding what boundary we're talking about. The kernel syscall interface? Things like Linux's vdso? POSIX library (libc)?
The more we go on in that sequence the more C we get into the interfaces and many of those APIs are essential for writing well-behaving non-trivial programs.
The closer we stay to the kernel's syscall interface the less C you get, but still the syscall interface and C are related. Strings are often NULL-terminated. Arguemtns are passed (via registers) from left to right, not like in Pascal convention right to left. (First arg always in EBX, second in EXC etc.)
A different language will need some interoperability with C conventions for being a good citizen on Unix.
A Pascal operating system for instance would use Pascal strings (historically what tis now called ShortString) where length is encoded in the first byte.
Only if the designers would make that decision, and they are not in any way obliged to make such decision! Nobody stops them from using e.g. a moral equivalent of "RECORD size: CARDINAL; str_ptr: ^CHAR END" for passing strings in their system calls (which would probably coincide with using "size_t size, char *str_ptr" in the C prototype at the binary level), or something else entirely.
Is the idea of an OS not exposing its implementation language in its public API that foreign so I have to express it three times in this thread and every time be replied with "No, what are you talking about, of course the OS would use reuse the structures of the runtime of the language that it is written in in its public syscall API"?
Gee, I wonder how the syscall calling convention would look if I wrote an OS entirely in assembly.
With C and Unix the issue of course is that both were developed around the same time by the same people for working together.
"All" languages (which survived) can call into C.
And yes, with common Pascal compilers i can write specific functions in a way to provide a C calling interface (called STDCALL) so those functions can be called from C. But for reaching arbitrary Pascal functions I need my own assembly code which passes parameters in the right order and manages the stack "correctly" (in C the caller clears the stack, in Pascal the calleé)
On x64 Linux, everyone managed to agree to use the same calling convention which is of course very convenient. The language of it's written specification is heavily C-biased, but if you squint you can see that it talks mostly about eightbytes and and compound objects and where to shove those so it more or less trivially translates to most other Algol- or C-descendant imperative languages; I believe it can even be reformulated strictly in language-neutral/assembly terms.
On vanilla x86, of course, everyone invented their own calling convention, even different implementations of the same languages: IIRC, Borland C++, MSVC, Watcom, and GCC on Windows all by default produced object files that you could not simply link together without manual tinkering. And (32-bit) Windows, despite being written in C, uses a variation of "Pascal" calling convention in its system API which they called "stdcall". I believe there were some C compilers that actually used "stdcall" by default instead of "cdecl".
Hell, even on x64, Cygwin GCC (on Windows) uses SysV ABI for everything except interfacing with Windows itself where it obviously has to use Windows x64 calling convention.
[0] http://www.gnu-pascal.de/gpc/Importing-Libraries-from-Other-...
But since C is the de facto standard for OS libraries, most languages end up supporting calls to C libraries. Thus C libraries do become the language-agnostic library.
So in order to have usable libraries people will wrap their C++ implementation in C interfaces. Rust faces the same problem. All those nice abstractions result in a complex ABI that will never touched by anything outside the Rust ecosystem. There is no escaping the extern "C".
Some languages are virtualized. Java, C#, Ruby, Python, Javascript all have virtual machines. There's no way to interface directly with code running in those languages. We must interface with the virtual machine instead: import it into our process and ask it to call the functions we want to use.
C binary interfaces are not language agnostic but they are quite simple. The result of compiling a C library is symbols pointing to functions with simple calling conventions. As a result, virtually any language can look up these symbols and call them.
The ideal solution is to have a system_call keyword built into the compiler that emits the proper instructions for the target platform.
https://github.com/torvalds/linux/blob/master/Documentation/...
On Windows and other operating systems, things such as system call numbers can and do change. There's no way to interface with the kernel without linking to vendor libraries.
So on Linux you can ditch libc and write your own code. A Rust liblinux is definitely possible, I tested it. You can even have a compiler generate the system calls for you! It just needs to support the simple calling convention. No libraries necessary. I've seen a project that does just that:
https://github.com/titzer/virgil
https://github.com/sunfishcode/mustang
It supports threads, filesystem, networking, and more!
The library for making Linux system calls is:
https://github.com/bytecodealliance/rustix
On Linux, however, we have, for example, glibc on one hand which claims that it doesn't have a goal of forwarding all of the Linux's syscall, and several failed initiatives on the other hand to allow the user's code to perform syscalls only from inside of glibc (which spells doom to anything that doesn't use libc, e.g. Golang) because of security reasons. And the history of still having to support INT 80h together with SYSCALL on x64 on the third hand.
I didn't know about these efforts. Please elaborate.
The basic idea is to make it harder for remote code execution bugs to result in useful abilities. Restricting access to syscalls to libc, and using adress randomization to make it hard to find libc makes it harder to do system calls when you take over a program.
https://lwn.net/Articles/806863/
> For static binaries, the valid regions are the base program's text segment and the signal trampoline page.
I'm not sure but it looks like the BSD libc is some kind a special case? Surely this is not necessary? A compiler or language runtime should be able to somehow mark those segments with the appropriate permissions.
The problem, again, is not that this kernel-entry interface is factored out into a separate shared library; the problem is that it's glued into a runtime of one specific programming language that now apparently everybody has to use.
[0] https://lwn.net/Articles/806776/
[1] https://news.ycombinator.com/item?id=21859612
Yeah. I don't like that. Just because that's the way it works now doesn't mean it must work that way forever. Because of its stable system calls, Linux doesn't impose any design on user space. People could theoretically rebuild the entire system in Lisp or Rust or something. Potential is limitless. Also, it's amazing that I can compile a freestanding program with zero dependencies and boot Linux straight into it.
> the problem is that it's glued into a runtime of one specific programming language that now apparently everybody has to use
I agree. It's a really hard problem to solve. Since it's based on the POSIX APIs there are tons of definitions and data types that are specified exclusively in C and its preprocessor. Not exactly accessible for other languages. The preprocessor macros in particular are extremely problematic. The only sane way to untangle those is to use a C compiler.
There's an interesting discussion in that thread you linked. What if the vDSO was the entry point for all system calls? That would certainly be interesting.
I think the vDSO ABI is underspecified. It's just the C ABI, whatever that means on your system. This can cause bugs due to assumptions. For example, this Go bug happened because Go called hardened versions of the vDSO functions with insufficient stack space:
https://news.ycombinator.com/item?id=28411786
Compared to the C ABIs, the Linux system call ABI is really simple and easy to understand. No way to screw it up.
The vDSO ABI should be as simple as the system call ABI. As it is right now, it pretty much forces people to use C.
You could always call syscalls directly, using syscall() (https://man7.org/linux/man-pages/man2/syscall.2.html), but a wrapper library is nicer.
One of my objectives was to produce ELFs with no symbols other than those defined in my own code. Glibc adds a lot of symbols.
Ohh, if only we could make C the second class citizen. C brings with itself meaningless legacy from 1969 into modern systems, like you might think openat2 was created recently, but it's designed for PDP-7, because it's C and stuff was always done this way.
Every Linux process interfaces through the kernel via the system call interface. It is stable and language agnostic. The interface is defined at the processor architecture level.
https://github.com/torvalds/linux/blob/master/Documentation/...
https://man7.org/linux/man-pages/man2/syscall.2.html
Any compiler can emit this sort of code, including JIT compilers! It's possible to discard the entire C user space and rewrite everything in Rust. Would be a monumental effort but still.
Note that no matter what there will be at least one C library remaining: the vDSO the kernel maps into the address space of every process.
https://github.com/torvalds/linux/blob/master/Documentation/...
https://news.ycombinator.com/item?id=25997506
Apparently for BSD, the ABI is not that stable, glibc works around bugs on some platforms (adding memory barriers etc), and there are security features that check the origin of syscalls. I wonder whether this is truly acceptable on Linux...
This is not a problem on Linux.
> glibc works around bugs on some platforms (adding memory barriers etc)
That's really interesting. Do you have examples of such fixes? I'm more familiar with musl sources, glibc proved to be somewhat difficult to navigate.
Well, yeah, but that's a tautology, no?
A willingness to discard everything indicates a willingness to rewrite everything. So, yeah, it is possible to rewrite everything if you are willing to rewrite everything.
Linux lets you do this. The interface is stable. People can build their custom user space on top of it and enjoy a great kernel full of drivers. Wouldn't it be amazing if someone came up with a 100% Lisp user space? A 100% Rust user space?
Great indeed (I like Lisp!), but I have to question how useful that would be in practice.
Let's take Lisp as an example: I can now program only in Lisp and talk to the OS in native Lisp only (including POSIX calls). The downsides are that I cannot call any existing library; my 100% Lisp environment cannot use libzip, or libffmpeg/libavutil, or libSDL, etc.
In order to make that 100% Lisp environment actually useful it's going to be doing FFI calls anyway into a C ABI. That makes the OS-interface redundant.
Python is as popular as it is purely because it made calling into the C ABI easy: existing libraries were made instantly useful in Python, not unnecessarily hard.
What replacement are you proposing that every language can call into? How would a migration of existing Java, Python, Ruby, C++ etc programs be done to this new interface? Do programs need to be modified at the source level? Just recompiled? How will the existing programs that cannot be recompiled be handled?
I quite aware that you can make the C ABI a shim into the new first class ABI to make existing programs work, but why would anyone target the new ABI when the existing one still works?
Okay, then it's aimed at people who want to use more than just POSIX, right? That's the entire world.
In which case the question still applies - how do you migrate everyone over to your new interoperable ABI? How will I run Gimp, Firefox/Thunderbird, IntelliJ, VSCode, Slack, Xterm, all my command-line tools (tar, grep, etc), Git, Meld, Starcraft 2 (and all 100+ games in my Steam library, and Steam itself) and more?
How do you see your replacement running MySQL, Apache, Nginx, PostgreSQL, bash/zsh/etc, SQLite, stunnel, netcat, wireshark/tcpdump, etc?
Do they all have to be recompiled? Replaced with new sootware?
Because each and every one of the software I listed above depends on a C ABI. And any shim provided to make existing software work will simply result in developers ignoring your new ABI and targeting the C ABI for maximum compatibility.
"access to a very limited subset of Linux system calls" :P Still a cool project though and a good base.