News for hackers. Stuff that matters.
Re:RipGrep musl binaries occasionally segfault during very-l... by Orphis on Saturday August 01, 2026 @10:57AM (#49135015)
That's good! But I wonder why it wasn't then enabled for that configuration or why the override wasn't "global" enough and the default allocator was still partly used.
Re:RipGrep musl binaries occasionally segfault during very-l... by tialaramex on Saturday August 01, 2026 @11:43AM (#49135395)
Rust doesn't get to override musl internals. Ripgrep uses opendir, a POSIX library feature implemented in musl to look at er, directories. The stack trace suggests we blew up when Rust's std::sys::fs::unix::readdir internal detail called opendir, and it in turn allocated.

On Linux it would be possible for ripgrep to talk directly to the kernel via documented system calls without libc, but that wouldn't work on any other popular OS.

Re:RipGrep musl binaries occasionally segfault during very-l... by inigyou on Saturday August 01, 2026 @01:52PM (#49136692)
I assume it already doesn't work on Windows. At some point you have to define your compatibility boundary. And high performance often coincides with mediocre compatibility.
Re:RipGrep musl binaries occasionally segfault during very-l... by galangalalgol on Saturday August 01, 2026 @02:04PM (#49136797)
A good example is go. On linux you can use a from scratch image fairly easily because it only uses syscalls. But for windows or mac the moving target wasn't maintainable so they link against shared objects. Linus enforcing the don't break userspace rule is what made that possible. That definitely has tradeoffs. At some point relibc or something similar will allow the same (stably) for rust. But using posix as that compatibility boundary gets you a much larger set of OS and only occasionally has a performance penalty. Often, a posix api tuned to the kernel is more performant. Musl is an exception precisely because it makes an openbsd-esque trade of performance for simple small attack surface.
Re:RipGrep musl binaries occasionally segfault during very-l... by inigyou on Saturday August 01, 2026 @02:15PM (#49136875)
The Linux kernel defines syscalls as its stable ABI (note: there are also non-kernel ABIs on Linux, such as Wayland) while Windows defines the DLL calls as its stable ABI.

One isn't better than the other. Linux's approach allows binaries to be fully statically linked, which is a more predictable environment for binaries, but Windows's approach composes better, as every process loads DLLs and this allows for things like graphics drivers and COM to work more reliably. As things stand on Linux you can't use the GPU in a portable statically linked app, because the kernel doesn't define the semantics of dynamic linking.

Re:RipGrep musl binaries occasionally segfault during very-l... by fc417fc802 on Saturday August 01, 2026 @07:35PM (#49139652)
TBF the GPU thing makes a fair amount of sense if you dig into it. It arguably falls entirely outside the kernel's domain of responsibility.

However given how fundamental GPU acceleration is (as well as various other pieces of dedicated hardware in various scenarios) it would be nice if the kernel defined some basic portable semantics for linking with important drivers. These could exist independently of libc and the rest of userspace purely as an optional fallback.

... or we could all just include a glibc compatible dlopen routine in our statically linked binaries instead of worrying about pedantic hypotheticals.

Re:RipGrep musl binaries occasionally segfault during very-l... by inigyou on Sunday August 02, 2026 @05:05AM (#49142560)
Yeah we could all just use glibc. You see the problem with that though right?
Re:RipGrep musl binaries occasionally segfault during very-l... by fc417fc802 on Sunday August 02, 2026 @10:45AM (#49145146)
That isn't at all what I said. I'm pointing out that in reality there aren't 20 competing libc implementations each with a unique dynamic linking scheme used by the userspace of 20 corresponding linux distributions. Instead we have ... glibc. So if the kernel actually specified a common portable linking scheme to fall back on as a means to interface with critical drivers it would presumably adopt the glibc way of doing things for that purpose.

Also I did not say "just use glibc" but rather suggested to just include a secondary glibc compatible dlopen routine. Exactly as you would have to do if the kernel bothered to specify.

Re:RipGrep musl binaries occasionally segfault during very-l... by inigyou on Sunday August 02, 2026 @10:50AM (#49145191)
If the kernel wanted to force you to use glibc it would do so. We could get rid of the stupid hardcoded paths in PT_INTERP for example.
Re:RipGrep musl binaries occasionally segfault during very-l... by fc417fc802 on Sunday August 02, 2026 @10:57AM (#49145255)
Right. Obviously. What I'm pointing out is that if the kernel decided to include dynamically linking to certain drivers within its scope (which it does not wish to do at present as you rightly point out) then it would need to specify a scheme for doing so. At which point glibc is all but a foregone conclusion.

So just pretend they did that and include a second dlopen routine and get on with life because at that point you're compatible with ~all major linux distros.

Far more pressing is a blessed filesystem path for where to find the things.

Re:RipGrep musl binaries occasionally segfault during very-l... by dataflow on Saturday August 01, 2026 @04:44PM (#49138294)
FYI, Windows's approach is better in some respects, like letting you avoid syscall overhead in some cases.
Re:RipGrep musl binaries occasionally segfault during very-l... by inigyou on Saturday August 01, 2026 @06:14PM (#49139010)
Notably, Linux has this too, but it's a special mechanism (vDSO) while on Windows it's just a normal function call with no difference from any other function.
Re:RipGrep musl binaries occasionally segfault during very-l... by dataflow on Saturday August 01, 2026 @10:48PM (#49140585)
Or in other words, they agreed the Windows approach was better, and tried to retrofit it (;
Re:RipGrep musl binaries occasionally segfault during very-l... by ChrisSD on Saturday August 01, 2026 @04:27PM (#49138127)
You can call into the kernel just fine in Windows. The fact you use a function call wrapper instead of a raw syscall is not really relevant.

Most Windows NT kernel functions take a buffer to use rather than allocating their own memory. Even many Win32 functions don't allocate (although there you have to be much more careful).

Re:RipGrep musl binaries occasionally segfault during very-l... by VorpalWay on Saturday August 01, 2026 @02:49PM (#49137222)
> Rust doesn't get to override musl internals.

Is malloc not a weak symbol in musl? I would expect it to be overridable like it is with glibc. Or does ripgrep only override Rust's global allocator?

Re:RipGrep musl binaries occasionally segfault during very-l... by masklinn on Saturday August 01, 2026 @03:34PM (#49137640)
> Or does ripgrep only override Rust's global allocator?

That one, as you can see in the code linked above, it only does rust-level overriding as that was where musl's allocator was found to impact ripgrep (https://github.com/BurntSushi/ripgrep/commit/03bf37ff4a29361...)

Re:RipGrep musl binaries occasionally segfault during very-l... by Orphis on Saturday August 01, 2026 @05:50PM (#49138825)
Last time I checked (a while ago), it wasn't. You needed to compile musl and exclude a few files from the archive to then provide your own allocator.

Depending on the toolchain, I'm assuming you could use some tricks or hacks to make it work better, but the tooling I used (Bazel) did that automatically and I haven't bothered to look at the internals yet .