News for hackers. Stuff that matters.
Re:A standalone zero-dependency Lisp for Linux by nanolith on Monday November 06, 2023 @10:58AM (#38164282)
This is interesting. Although, I think it's always useful to point out that runtimes that directly make system calls will have portability issues in other *nix systems. For instance, OpenBSD has recently restricted system calls to a single offset of their libc in order to reduce ROP attacks.

If Lone wishes to be portable, it will need to consider a libc dependency as an option. If not, it can probably get away with direct syscalls on Linux unless Linux kernel developers decide to add support for pinning system calls. I doubt that this would ever be a hard requirement in the kernel, as it would break userland all over, but I could see this being an option for certain distributions as a ROP mitigation. Many features like these flow from OpenBSD to other OSes.

Re:A standalone zero-dependency Lisp for Linux by edgyquant on Monday November 06, 2023 @11:19AM (#38164582)
Considering the title and use case it seems to be intentionally pretty specific to Linux
Re:A standalone zero-dependency Lisp for Linux by nanolith on Monday November 06, 2023 @01:15PM (#38166381)
Many projects start this way. But, as per my comment, the assumption that direct syscall support will be maintained in future Linux distros is also risky.
Re:A standalone zero-dependency Lisp for Linux by matheusmoreira on Monday November 06, 2023 @03:08PM (#38168202)
I worry about that risk as well. I assume that even if Linux were to introduce a mechanism for system call authentication, it would be something lone would be able to use to mark its system call primitive as allowed.
Re:A standalone zero-dependency Lisp for Linux by nanolith on Monday November 06, 2023 @03:14PM (#38168304)
Perhaps. To be fair, I'm not aware of anything on the horizon, other than the fact that OpenBSD has been showing off their pinning implementation.

As long as you know it's a possibility, then the point of my original comment is met.

Good luck on this project. I look forward to seeing it progress.

Re:A standalone zero-dependency Lisp for Linux by matheusmoreira on Monday November 06, 2023 @03:32PM (#38168553)
Thank you!
Re:A standalone zero-dependency Lisp for Linux by kwhitefoot on Monday November 06, 2023 @03:47PM (#38168759)
Why is it risky? Linus is adamant that the greatest sin is breaking userland.
Re:A standalone zero-dependency Lisp for Linux by nanolith on Monday November 06, 2023 @04:24PM (#38169224)
That's a subtle point though. The kernel can't change defaults that break userland, nor can it change or eliminate features that would cause a breakage in userland. But, the kernel can certainly add an optional feature, like syscall pinning, that distributions can enable -- in userland -- to restrict userland. We see this already with seccomp policies meant to restrict and potentially break userland programs that misbehave.

All that Linus guarantees is that, by default, the Linux kernel has no regressions that impact user code. If distributions enable breaking changes through syscalls or sysctls, that doesn't violate any of the rules imposed on Linux. syscall pinning -- if that becomes a thing in Linux -- is something that distributions would enable in order to mitigate ROP attacks.

Re:A standalone zero-dependency Lisp for Linux by Conscat on Monday November 06, 2023 @01:28PM (#38166608)
I think this decision from OpenBSD will more likely discourage developers from even considering it a supportable platform for software that originates in Linux land.
Re:A standalone zero-dependency Lisp for Linux by nanolith on Monday November 06, 2023 @01:46PM (#38166884)
Direct syscall access is not something that is guaranteed in Unix derivatives. Linux is rare in that it provides a stable syscall API. Source compatibility is often only guaranteed when linked against libc or an equivalent low-level runtime library.
Re:A standalone zero-dependency Lisp for Linux by eikenberry on Monday November 06, 2023 @01:52PM (#38166983)
And this is generally a bad pattern unless those libc equivalents are services you call (like syscalls) and not a library you have to import or FFI. Requiring importing a library, probably from another language, is not a good alternative to syscalls.
Re:A standalone zero-dependency Lisp for Linux by nanolith on Monday November 06, 2023 @02:51PM (#38167927)
A bad pattern according to whom? Most language runtime libraries import other system libraries as needed. For better or for worse, libc is typically considered to be a system library. It's something that every distribution or Unix flavor provides that is guaranteed to work within the POSIX standard for interfacing with the operating system. It's up to the distribution maintainers to make that happen, even if they tweak things to support syscall pinning or seccomp rules.

Userland directly calling a stable syscall API is a rare thing outside of Linux, and there is no guarantee that it will last forever even in Linux given the latest attacks. With modern ROP mitigations like syscall pinning, it will in fact be more dangerous to make syscalls directly -- if allowed in your distribution -- than it would be to call the minimal footprint of libc required to bootstrap a high level language runtime.

Of course, with special pleading, it could be possible for distribution or OS maintainers to carve out an exception for syscall pinning for a particular language runtime. Ask Go how that's going for their OpenBSD port.

Re:A standalone zero-dependency Lisp for Linux by eikenberry on Monday November 06, 2023 @03:27PM (#38168494)
The problem with system libraries requiring importing them as C libraries isn't new and doesn't seem to be going away. It has caused all sorts of problems for alternative languages over the years that it seems like an alternative would give all of computing a giant boost by allowing different models that don't work well with C. Stabilizing and standardizing the syscall interfaces would be one way to accomplish this and is the closest thing we have to it now. Implementing syscalls as a separate service might also work but then you have the IPC overhead. That might not be as bad though as we'll end up with something like that anyways as requirements ramp up for C to have it's own runtime (https://dslab.epfl.ch/research/cpi/).
Re:A standalone zero-dependency Lisp for Linux by nanolith on Monday November 06, 2023 @04:00PM (#38168928)
For most language runtimes, the minimal requirement for libc integration is to cover the standard Unix calls (unistd) which don't require specific memory management and typically just pass buffers directly from the caller to the kernel. For most of the system calls in which a high level language runtime would be interested, the libc code is largely a direct pass-through already. As such, either directly using the system calls or calling them through libc will have negligible impact on how the high level language chooses to model these things.

libc isn't really getting in the way here.

Perhaps POSIX might come up with an alternative library to wrap system calls in the future, but I would suspect that it would probably be written in C on most platforms, or at least using a C compatible ABI. So, even if a platform chose to use Rust with a large littering of unsafe all over to make it work with the kernel, it would still have to be able to be linked with C userland.

Re:A standalone zero-dependency Lisp for Linux by matheusmoreira on Monday November 06, 2023 @10:27PM (#38172862)
> libc isn't really getting in the way here.

It depends. For the standard set of system calls, the libc is pretty great. For Linux-specific features, it could take years for glibc to gain support. All the libcs will get in the way if we want to use something like the clone system call.

My obsession with Linux system calls started years ago when I read about an episode where glibc literally got into Linux's way. The LWN wrote extensively about the tale of the getrandom system call and the quest to get glibc to support it:

https://lwn.net/Articles/711013/

A kernel hacker wrote in an email:

> maybe the kernel developers should support a libinux.a library that would allow us to bypass glibc when they are being non-helpful

That made a lot of sense to me. I took that concept and kind of ran with it. Started a liblinux project, essentially a libc with nothing in it but the thinnest possible system call wrappers. Researched quite a bit about glibc's attitude towards Linux to justify it:

https://github.com/matheusmoreira/liblinux#why

The more I used this stuff, the more I enjoyed it. I was writing freestanding C and interfacing directly with the kernel. The code was so clean. No libc baggage anywhere, not even errno. And I knew this could do literally anything when I wrote a little freestanding program to print my terminal window dimensions. When I did that I knew I could write code to mount disks too if I really wanted to. I was talking to the kernel.

Eventually I discovered Linux was already doing the same thing with their own nolibc.h file which they were already using in their own tools. It was a single file back then, by now it's become a sprawling directory full of code:

https://github.com/torvalds/linux/tree/master/tools/include/...

Even asked Greg Kroah-Hartman on reddit about it once:

https://old.reddit.com/r/linux/comments/fx5e4v/im_greg_kroah...

Since the kernel was already developing their own awesome headers, I decided to drop liblinux and start lone instead. :)

Re:A standalone zero-dependency Lisp for Linux by johncowan on Tuesday November 07, 2023 @01:15AM (#38173792)
> there is no guarantee that [syscall stability] will last forever even in Linux given the latest attacks

That's true, but what of it? Linus won't last forever, Linux won't last forever, computers won't last forever, and Homo sapiens won't last forever. Everything needs maintenance sooner or later. "The Rockies may crumble / Gibraltar may crumble / They're only made of clay."

Re:A standalone zero-dependency Lisp for Linux by nanolith on Tuesday November 07, 2023 @09:40AM (#38177253)
What you say is true, but you've inserted an inaccurate context with the quote.

There is no guarantee that _direct access to system calls_ will last forever...

Stability in the syscall API exists because Linux is a kernel that supports multiple distributions. Not because random applications could call it. The latter is an emergent feature, but not one that distribution maintainers will necessarily respect. POSIX only guarantees access to functions in libc that can perform these calls. As mentioned elsewhere in this thread, there are specific reasons why future direct access to system calls in user code could be restricted. Whether they will or not comes down to how distribution maintainers decide to deal with syscall related ROP gadgets.

Re:A standalone zero-dependency Lisp for Linux by dontlaugh on Monday November 06, 2023 @02:59PM (#38168047)
No, it’s fine. And common, macOS and Windows do the same.
Re:A standalone zero-dependency Lisp for Linux by baq on Monday November 06, 2023 @03:26PM (#38168476)
To be fair, linking to kernel32 is a bit different than linking to msvcrt, but yeah, it’s Linux who’s the slightly insane person in the room, not the other way around.
Re:A standalone zero-dependency Lisp for Linux by dontlaugh on Monday November 06, 2023 @03:32PM (#38168560)
I wouldn’t say insane exactly, but definitely overly restrictive.
Re:A standalone zero-dependency Lisp for Linux by snvzz on Monday November 06, 2023 @09:04PM (#38172243)
What decision from OpenBSD?

Note that Linux is the odd one here. Every other relevant system out there is not like Linux on this.

Re:A standalone zero-dependency Lisp for Linux by matheusmoreira on Monday November 06, 2023 @03:02PM (#38168105)
You're absolutely right, it is not portable to other operating systems. I have written about this portability and system call stability here on HN a few times, at some point I decided to compile all the information into a post on my website.

https://www.matheusmoreira.com/articles/linux-system-calls

I started lone (and previously a liblinux library) in order to make applications directly targeting the Linux system call binary interface. I chose Linux precisely because it's the only one with a stable interface.

I currently have no plans to make lone a portable programming language.