News for hackers. Stuff that matters.
Re:OpenBSD System-Call Pinning by yobert on Friday February 02, 2024 @05:21PM (#39235239)
> The direct-syscalls-inside-the-binary model used by go (and only go, noone else in the history of the unix software does this) provided the biggest resistance against this effort".

I know this annoys unix people. But I have to say I actually really like that Go shakes this up. I believe the C function monopoly just isn't healthy for things. You should be able to make a new completely unrelated language. The Go developers were the first in a long time to do this, not because they are stupid, but because they were ambitious.

Re:OpenBSD System-Call Pinning by anonacct37 on Friday February 02, 2024 @05:44PM (#39235475)
I do like this about go. And on Linux it arguably makes sense (I say arguable because DNS without CGO is still a common cause of issues and incompatibility).

But Linux has, to the best of my understanding said "yes, we are ok with users using syscalls". Linux doesn't think that glibc is the only project allowed to interface with the kernel.

But for other platforms like OpenBSD and windows they are quite simply relying on implementation details that the vendors consider to be a private and unsupported interface.

This whole thing is also separate from "is making libc the only caller of the syscalls instruction" a good and meaningful security improvement.

Re:OpenBSD System-Call Pinning by p_l on Friday February 02, 2024 @06:10PM (#39235728)
Windows also provides language-independent ways of calling low-level (and pretty high-level) OS interfaces.

The library you have to link to access system services is not going to pollute your language environment with bad runtime.

Re:OpenBSD System-Call Pinning by akira2501 on Friday February 02, 2024 @06:35PM (#39235994)
> I say arguable because DNS without CGO is still a common cause of issues and incompatibility

DNS without CGO works perfectly. The vendor specific ad hoc mechanisms for extending DNS in a site local context are not well supported. If they were implemented more sensibly, then Go, or any other language, would have no problem taking advantage of them even without the "C Library Resolver."

Speaking of which, that "C Library Resolver," in my opinion, has one of the worst library interfaces in all of unix. It's not at all a hill worth new projects dying on.

Re:OpenBSD System-Call Pinning by anonacct37 on Friday February 02, 2024 @06:56PM (#39236161)
> DNS without CGO works perfectly

It does not. I know this because it impacts my daily work and the work of others. Honestly if you could make my day and go figure out exactly what's going wrong with the pure go DNS implementation it would make my life alot simpler and I wouldn't have to maintain shell scripts that update etc/hosts to hard code in ipv4 addresses for the APIs I access with terraform.

https://github.com/hashicorp/terraform-provider-google/issue...

Re:OpenBSD System-Call Pinning by akira2501 on Friday February 02, 2024 @08:31PM (#39236780)
It seems like the explanation might be right there in that issue. The server is occasionally not successfully sending the A record and you're getting bad fallback behavior due to the way the dial call is imprecisely constructed and this is masking the underlying problem. The code, as written, is doing exactly what you would expect in this scenario.

Should it be your DNS resolver library that takes stock of your OS environment and only make calls for A records and not AAAA records when it "detects" some configuration?

Shouldn't your application itself have an environment variable or command line option that allows you to specify that your dials should only be done using tcp4? Wouldn't this be immensely useful to have outside of "auto detection" in some library somewhere?

Re:OpenBSD System-Call Pinning by j16sdiz on Saturday February 03, 2024 @12:08AM (#39237687)
Oh.. right. So we need another configuration for each and every application for something almost always system wide and can change dynamically (like wifi reconnecting) ?
Re:OpenBSD System-Call Pinning by tsimionescu on Saturday February 03, 2024 @02:52AM (#39238342)
Why should every application be aware of whether it is running on a v4 or v6 network? If the application merely wants to connect to an external service, that is firmly in the OS's job to decide.

As a user, if ipv6 is flaky today, I want one central place to configure for ipv4 only, I don't want to go and change every application'settings only to revert that tomorrow.

Re:OpenBSD System-Call Pinning by ninkendo on Friday February 02, 2024 @10:44PM (#39237358)
> The vendor specific ad hoc mechanisms for extending DNS in a site local context are not well supported

DNS is one of those things that OS vendors think should be extendable and configurable. It allows VPN apps to redirect DNS only for certain subdomains, for example, which enables proper split-horizon DNS. I think this is totally reasonable behavior, and it’s undeniably useful. If a particular programming language reimplements DNS on its own, you lose guarantees that the OS is striving to provide to the user.

You can make the case that OS’s shouldn’t make these guarantees, and we’re free to disagree on that, but from a practical standpoint it is a very useful feature and it sucks that pure Go apps don’t work with it.

Re:OpenBSD System-Call Pinning by wbl on Saturday February 03, 2024 @02:04AM (#39238165)
The right way to do this is specify a resolver to use on localhost
Re:OpenBSD System-Call Pinning by treffer on Saturday February 03, 2024 @08:08AM (#39240015)
Or you make the case that the OS should ship a DNS server. It can handle DNS forwarding and iterative queries anyway (iterative queries as you often need them for dnssec) and the logic for serving DNS responses is ok'ish (DNS name dedup and length restrictions being the biggest complexity issue).

I am not super happy with systemd-resolved but it solves this particular issues. No requirement to use libc, but same (os configurable) behavior for all users.

Re:OpenBSD System-Call Pinning by tptacek on Saturday February 03, 2024 @11:14AM (#39241698)
The difficulty and overhead of running those queries from everybody's phones is one of the reasons (a small reason, but one of them) DNSSEC isn't deployed.
Re:OpenBSD System-Call Pinning by toast0 on Friday February 02, 2024 @08:55PM (#39236901)
Most unixes allow for static binaries. And most unixes allow for you to run a static binary compiled for an older OS on a newer OS. In order for that to work, the syscall interface needs to be at least mostly stable.

Yes, if you're writing netstat or lsof or ps or something, you need tight coupling with the binary and the kernel, and you can argue Linux does that better, but most people aren't writing netstat or lsof or ps.

Re:OpenBSD System-Call Pinning by IAmLiterallyAB on Saturday February 03, 2024 @12:38AM (#39237821)
Even those get most of their info from /proc and /sys rather than special syscalls
Re:OpenBSD System-Call Pinning by epcoa on Saturday February 03, 2024 @02:25AM (#39238219)
On the BSDs they sure don't, /proc (for its role referred to) and /sys are peculiar to Linux, /proc is not even mounted by default on FreeBSD, is deprecated and doesn't exist on OpenBSD. On OpenBSD/FreeBSD most of the info comes out with sysctl(3) or reading kernel memory directly.
Re:OpenBSD System-Call Pinning by saagarjha on Saturday February 03, 2024 @05:19AM (#39239034)
> reading kernel memory directly

God I hope not

Re:OpenBSD System-Call Pinning by epcoa on Saturday February 03, 2024 @08:11AM (#39240037)
https://man.freebsd.org/cgi/man.cgi?query=kvm&sektion=3&n=1

The aforementioned tools use these interfaces.

Most of the sundry information for the live kernel comes from sysctl (abstracted by kvm), I note the mem/kmem interface for completeness.

Re:OpenBSD System-Call Pinning by adrian_b on Saturday February 03, 2024 @07:16AM (#39239621)
I do not know what the previous poster was talking about, but, while unrestricted access to kernel memory is of course unacceptable, a perfectly secure and useful means for the kernel to provide information to a user process is to map a page as read-only in the user address space, from where the user process could get the information without the overhead of a system call.

For information of general interest such a special kernel page could be mapped as read-only in the address space of all user processes.

Much of the information that is provided in the special file systems /proc and /sys could have been provided in some appropriate data structures in such read-only shared memory, for a faster access, by avoiding the overhead of file system calls and of file text parsing.

Re:OpenBSD System-Call Pinning by Varriount on Saturday February 03, 2024 @01:41AM (#39238077)
Does Go use direct syscalls on Windows? Last time I looked, I could have sworn Go's source used the standard Windows system DLLs.
Re:OpenBSD System-Call Pinning by pjmlp on Saturday February 03, 2024 @03:25AM (#39238498)
On Windows it never did, and on other platforms, other than Linux, it has been forced to accept that isn't the way OS vendors play the game.
Re:OpenBSD System-Call Pinning by cryptonector on Friday February 02, 2024 @05:44PM (#39235476)
This isn't about Go vs C. This is about about indirection of system calls through stubs and also in a way about static linking vs. dynamic linking. Something like this could be done for C and others, but the kernel would have to walk the C stack to ensure that a) the callers of the system calls are run-time library stubs listed in the ELF, and b) that the callers of the stubs are listed in the ELFs (plural). Clearly this is easier to do if there's neither stubs nor dynamic linking, but that's not a reason to think Go is somehow superior to the rest.
Re:OpenBSD System-Call Pinning by steveklabnik on Friday February 02, 2024 @05:29PM (#39235321)
My hot take here is that this split exists because of the way that Linux development works differently than other Unices and other operating systems. Essentially, it's an example of Conway's Law.

As always, things are about API boundaries: what is internal to you, and what is exposed externally to your users.

With Linux, well, we all know the rms copypasta: "I'd just like to interject for a moment. What you're referring to as Linux, is in fact, GNU/Linux, or as I've recently taken to calling it, GNU plus Linux." This is a joke, but it also points to something real and serious: Linux, being just a kernel, means that they provide a stable kernel API. glibc, being written by a different organization, builds on top of that API and adds its own stable API.

By contrast, many other operating systems are developed as a full operating system. They don't produce a kernel as a standalone component. As such, they choose some other sort of boundary as the API for the OS. On unices, that's often the libc. On Windows, that's the Windows API, of which win32 is an example.

There are good reasons to not make the kernel your API boundary. Different systems make different choices, and that's a good thing, not a bad thing.

Re:OpenBSD System-Call Pinning by akira2501 on Friday February 02, 2024 @06:37PM (#39236009)
> There are good reasons to not make the kernel your API boundary.

Good reasons for the developers, not so much for the end users.

Re:OpenBSD System-Call Pinning by SSLy on Friday February 02, 2024 @07:09PM (#39236250)
WinNT seems to do just fine with that. And it's only because of users insisting on using abandonware
Re:OpenBSD System-Call Pinning by j16sdiz on Saturday February 03, 2024 @12:26AM (#39237753)
Developers works on win32api, not the kernel side NT* whatever stuffs.
Re:OpenBSD System-Call Pinning by SSLy on Saturday February 03, 2024 @05:42AM (#39239132)
Yes, that's my point. App developers (actual users of the win32 ABI) seem just fine with not calling kernel directly. App users also enjoy best in world backcompat.
Re:OpenBSD System-Call Pinning by steveklabnik on Friday February 02, 2024 @06:42PM (#39236051)
I don't agree that those two things are at odds.
Re:OpenBSD System-Call Pinning by devit on Friday February 02, 2024 @06:40PM (#39236029)
Anything statically linked to the C library will also have direct syscalls in the binary.
Re:OpenBSD System-Call Pinning by Rusky on Saturday February 03, 2024 @12:39AM (#39237824)
I don't think syscall-vs-C-function is at all meaningful for someone making a completely unrelated language. The system has to pick some ABI for communicating with the kernel, and syscalls don't change that. The ISA generally doesn't define how arguments are passed, it's really just in the business of providing a fancy jump instruction.

Arguably, using the same ABI for userspace C functions and for communicating with the kernel reduces the amount of work required of completely new languages, because they are likely to need C interop support anyway.

Re:OpenBSD System-Call Pinning by touisteur on Saturday February 03, 2024 @08:28AM (#39240153)
If you're designing a whole new language today, you might want to build something to fit better with the new io_uring-like interfaces to the kernel and make 'old school' syscalls a fallback or special case. Would be interesting to have this (queueing, batching, completion) as a basic layer.
Re:OpenBSD System-Call Pinning by mike_hock on Saturday February 03, 2024 @02:23AM (#39238211)
Yes, I'd like the OpenBSD approach a lot more if the specially privileged syscall library was a language-agnostic library just for syscalls instead of libc.

Also, if you're gonna add new ELF sections anyway, why not do syscall "relocation" directly (with a similar randomization like ASLR) instead of going through stubs? This "relocation" doesn't actually need to change any memory or use any offset tables since the syscall numbers are a farce under the new system anyway. Just make it a map of the location of syscall instructions to implied syscall numbers. Once you've phased out the old syscall model, you can even repurpose RAX as an additional syscall parameter.

Re:OpenBSD System-Call Pinning by Someone on Saturday February 03, 2024 @08:05AM (#39239997)
> I'd like the OpenBSD approach a lot more if the specially privileged syscall library was a language-agnostic library just for syscalls instead of libc.

I think they should split that part of, too. Such a split would better reflect the split of responsibilities between kernel maintainers and libc maintainers.

At the same the effective difference for people using the code is minute.

Any decent linker will strip out unused functions in libraries you link with, so if you currently only use the part of libc that would become “libsyscall”, the end effect would be the same as when “libsyscall” existed.