News for hackers. Stuff that matters.
Re:liblinux: Architecture-independent access to Linux system... by GoblinSlayer on Tuesday November 09, 2021 @03:37AM (#29159138)
>The C library is itself a user space program, so the rule that a user space program has to go through the C library obviously isn't logical. And the GNU C library is an optional tool, not an official part of the kernel interface (if it were the latter, I would expect to see it packaged with the kernel).

Ohh, if only we could make C the second class citizen. C brings with itself meaningless legacy from 1969 into modern systems, like you might think openat2 was created recently, but it's designed for PDP-7, because it's C and stuff was always done this way.

Re:liblinux: Architecture-independent access to Linux system... by matheusmoreira on Tuesday November 09, 2021 @08:47AM (#29161634)
With Linux, you can! I started this project years ago just to prove to myself that it was possible.

Every Linux process interfaces through the kernel via the system call interface. It is stable and language agnostic. The interface is defined at the processor architecture level.

https://github.com/torvalds/linux/blob/master/Documentation/...

https://man7.org/linux/man-pages/man2/syscall.2.html

Any compiler can emit this sort of code, including JIT compilers! It's possible to discard the entire C user space and rewrite everything in Rust. Would be a monumental effort but still.

Note that no matter what there will be at least one C library remaining: the vDSO the kernel maps into the address space of every process.

https://github.com/torvalds/linux/blob/master/Documentation/...

Re:liblinux: Architecture-independent access to Linux system... by remram on Tuesday November 09, 2021 @11:21AM (#29163743)
Isn't this the route Golang famously went, with poor results?

https://news.ycombinator.com/item?id=25997506

Apparently for BSD, the ABI is not that stable, glibc works around bugs on some platforms (adding memory barriers etc), and there are security features that check the origin of syscalls. I wonder whether this is truly acceptable on Linux...

Re:liblinux: Architecture-independent access to Linux system... by matheusmoreira on Tuesday November 09, 2021 @01:30PM (#29165413)
It is my understanding that Go attempted to perform direct system calls on systems where it was never supported. As a result, Go programs stopped working when the binary interface changed.

This is not a problem on Linux.

> glibc works around bugs on some platforms (adding memory barriers etc)

That's really interesting. Do you have examples of such fixes? I'm more familiar with musl sources, glibc proved to be somewhat difficult to navigate.

Re:liblinux: Architecture-independent access to Linux system... by lelanthran on Wednesday November 10, 2021 @01:44AM (#29171807)
> Any compiler can emit this sort of code, including JIT compilers! It's possible to discard the entire C user space and rewrite everything in Rust.

Well, yeah, but that's a tautology, no?

A willingness to discard everything indicates a willingness to rewrite everything. So, yeah, it is possible to rewrite everything if you are willing to rewrite everything.

Re:liblinux: Architecture-independent access to Linux system... by matheusmoreira on Wednesday November 10, 2021 @09:51AM (#29175187)
A willingness to discard everything doesn't mean you can actually do that though. Windows, for example, simply doesn't let you get rid of the Microsoft DLLs. You can't talk to the kernel directly, the interface will change and break your program. The BSDs and macOS all do the same thing.

Linux lets you do this. The interface is stable. People can build their custom user space on top of it and enjoy a great kernel full of drivers. Wouldn't it be amazing if someone came up with a 100% Lisp user space? A 100% Rust user space?

Re:liblinux: Architecture-independent access to Linux system... by lelanthran on Thursday November 11, 2021 @05:37AM (#29186529)
> Linux lets you do this. The interface is stable. People can build their custom user space on top of it and enjoy a great kernel full of drivers. Wouldn't it be amazing if someone came up with a 100% Lisp user space? A 100% Rust user space?

Great indeed (I like Lisp!), but I have to question how useful that would be in practice.

Let's take Lisp as an example: I can now program only in Lisp and talk to the OS in native Lisp only (including POSIX calls). The downsides are that I cannot call any existing library; my 100% Lisp environment cannot use libzip, or libffmpeg/libavutil, or libSDL, etc.

In order to make that 100% Lisp environment actually useful it's going to be doing FFI calls anyway into a C ABI. That makes the OS-interface redundant.

Python is as popular as it is purely because it made calling into the C ABI easy: existing libraries were made instantly useful in Python, not unnecessarily hard.

Re:liblinux: Architecture-independent access to Linux system... by lelanthran on Tuesday November 09, 2021 @04:54AM (#29159624)
What would be the first class citizen after you make C second class?
Re:liblinux: Architecture-independent access to Linux system... by GoblinSlayer on Tuesday November 09, 2021 @05:46AM (#29159975)
More precisely it's about making PDP-7 design the second class citizen. Interoperable design becomes the first class instead. POSIX would be implemented on top of that.
Re:liblinux: Architecture-independent access to Linux system... by lelanthran on Tuesday November 09, 2021 @06:24AM (#29160229)
I understand that, but right now the C ABI is first class, and every language can call into it.

What replacement are you proposing that every language can call into? How would a migration of existing Java, Python, Ruby, C++ etc programs be done to this new interface? Do programs need to be modified at the source level? Just recompiled? How will the existing programs that cannot be recompiled be handled?

I quite aware that you can make the C ABI a shim into the new first class ABI to make existing programs work, but why would anyone target the new ABI when the existing one still works?

Re:liblinux: Architecture-independent access to Linux system... by GoblinSlayer on Tuesday November 09, 2021 @10:30AM (#29162999)
Obviously it's not aimed at people who want to use only POSIX.
Re:liblinux: Architecture-independent access to Linux system... by lelanthran on Tuesday November 09, 2021 @01:23PM (#29165306)
> Obviously it's not aimed at people who want to use only POSIX.

Okay, then it's aimed at people who want to use more than just POSIX, right? That's the entire world.

In which case the question still applies - how do you migrate everyone over to your new interoperable ABI? How will I run Gimp, Firefox/Thunderbird, IntelliJ, VSCode, Slack, Xterm, all my command-line tools (tar, grep, etc), Git, Meld, Starcraft 2 (and all 100+ games in my Steam library, and Steam itself) and more?

How do you see your replacement running MySQL, Apache, Nginx, PostgreSQL, bash/zsh/etc, SQLite, stunnel, netcat, wireshark/tcpdump, etc?

Do they all have to be recompiled? Replaced with new sootware?

Because each and every one of the software I listed above depends on a C ABI. And any shim provided to make existing software work will simply result in developers ignoring your new ABI and targeting the C ABI for maximum compatibility.