News for hackers. Stuff that matters.
Re:Porting Rust's Std to Rustix by codeflo on Tuesday January 04, 2022 @07:37AM (#29794068)
As a believer in removing unnecessary layers of abstraction, this looks interesting. Is there a technical reason Rustix doesn’t support Windows, or is that just not a priority at the moment?

(For a bit of background info: It might seem counterintuitive, but on Windows, a “native” application “should” actually call the programming language agnostic Kernel32.dll functions directly. At least that’s the documented, stable way of doing things. Instead, Rust’s std currently goes through libc, which is also fine, but on Windows is an abstraction built on top of the Windows APIs, and is historically a lot less stable. This can cause code bloat and distribution hassles.

This is a bit different from Linux, where the abstractions are layered the other way round: the OS (POSIX) spec mostly assumes an existing libc. As a result, on Linux, going libc-less is a bit harder in my limited experience — though certainly possible if you know what you’re doing.)

Re:Porting Rust's Std to Rustix by sanxiyn on Tuesday January 04, 2022 @08:17AM (#29794360)
> Instead, Rust's std currently goes through libc

Are you sure about this? Rust's std's File::open calls CreateFile on Windows, not libc open. Isn't CreateFile correct Kernel32.dll API to call?

https://github.com/rust-lang/rust/blob/master/library/std/sr...

Re:Porting Rust's Std to Rustix by codeflo on Tuesday January 04, 2022 @08:45AM (#29794631)
It is. I haven’t checked the complete source code. What I know for sure is that a program using Rust’s std still requires the C runtime to link and run, and from that perspective, it doesn’t really matter if some or even many parts of the std don’t actually use it.
Re:Porting Rust's Std to Rustix by sanxiyn on Tuesday January 04, 2022 @09:01AM (#29794789)
I see. I wonder where Rust's std is using libc on Windows. I know for a fact filesystem portion of std doesn't call libc at all on Windows.
Re:Porting Rust's Std to Rustix by ChrisSD on Tuesday January 04, 2022 @09:03AM (#29794815)
Rust needs the C runtime for startup/shutdown code (and vcruntime for panics). It also needs basic memory functions such as memcpy and stack probes. There is no way round this except rewriting those in Rust.
Re:Porting Rust's Std to Rustix by sanxiyn on Tuesday January 04, 2022 @09:23AM (#29795014)
Rust already ships with its own memcpy, so it should be possible: https://github.com/rust-lang/compiler-builtins/blob/master/s...
Re:Porting Rust's Std to Rustix by ectopod on Tuesday January 04, 2022 @12:51PM (#29797614)
The startup/shutdown code is for the C runtime. If you avoid the C runtime you don't need it.
Re:Porting Rust's Std to Rustix by ChrisSD on Tuesday January 04, 2022 @01:48PM (#29798418)
At a minimum Rust would have to run the C initializers as these are used even in pure Rust code. It might also make use of the security cookie but I'm uncertain about that. Also the C runtime is needed by the SEH handling code (in vcruntime) so Rust would have to replace that before it can replace the C startup/shutdown.

To be clear, this is feasible but it'll require someone knowledgable to put in the work of rewriting this in Rust.

Re:Porting Rust's Std to Rustix by pjmlp on Wednesday January 05, 2022 @03:13AM (#29806431)
SEH doesn't need the C runtime, they are only required when using the C language extensions for frame-based Exception Handling, with the new vectored Exception Handling, plain Win32 calls will do,

https://docs.microsoft.com/en-us/windows/win32/debug/vectore...

Granted, they are more cumbersome to use and probably not worth the effort to try to avoid them other than special cases.

Re:Porting Rust's Std to Rustix by ChrisSD on Wednesday January 05, 2022 @05:52AM (#29807494)
Sure. I did not mean to imply that SEH itsef requires the C runtime. However, the __CxxFrameHandler3 that Rust/llvm uses would need to be rewritten.

IIRC llvm itself generates calls to __CxxFrameHandler3 when SEH is being used, making it awkward to replace with something that's not compatible.

Re:Porting Rust's Std to Rustix by steveklabnik on Tuesday January 04, 2022 @01:56PM (#29798525)
https://crates.io/crates/r0 exists (not everything you're talking about but like, there's some of this stuff around, in some contexts. We'll see if the whole pile of stuff needed ever gets ported or not, I'm guessing yes but on a long timeframe.)
Re:Porting Rust's Std to Rustix by ylyn on Tuesday January 04, 2022 @08:27AM (#29794438)
> This is a bit different from Linux, where the abstractions are layered the other way round: the OS (POSIX) spec mostly assumes an existing libc.

FWIW the POSIX spec assumes libc, but on Linux specifically the syscall interface is stable.

Re:Porting Rust's Std to Rustix by sanxiyn on Tuesday January 04, 2022 @08:30AM (#29794463)
Note that macOS syscall interface is specifically unstable so Rust needs POSIX/libc path anyway.
Re:Porting Rust's Std to Rustix by masklinn on Tuesday January 04, 2022 @09:23AM (#29795016)
Also on Solaris/Illumos/SmartOS/..., and on OpenBSD (where it's more or less verboten entirely since system-call-origin verification).
Re:Porting Rust's Std to Rustix by wahern on Tuesday January 04, 2022 @07:06PM (#29802888)
AFAIU, you don't need to use libc on OpenBSD, however only one contiguous region of code is permitted to invoke syscalls, and so you can't mix-and-match ad hoc libraries. See https://man.openbsd.org/msyscall, which implements one-time registration of a contiguous range of pages permitted to invoke syscalls.
Re:Porting Rust's Std to Rustix by codeflo on Tuesday January 04, 2022 @08:56AM (#29794737)
Thanks, that’s a great correction/clarification.
Re:Porting Rust's Std to Rustix by samhw on Tuesday January 04, 2022 @07:43AM (#29794106)
Is there a reason "native" and "should" are in scare quotes? I think "should" is pretty well defined - after all, we've got an RFC for it!

Edit: I'm not altogether sure why this is being downvoted, but in case it wasn't clear: this is a sincere question and not some kind of pedantic rhetorical question. (I don't really mind whether people upvote or downvote the comment, except to the extent that it's a signal that I perhaps wasn't clear in what I was asking.)

Re:Porting Rust's Std to Rustix by CodesInChaos on Tuesday January 04, 2022 @09:44AM (#29795258)
There isn't anything objectively wrong with using libc as an abstraction on Windows, especially since Win10 ships the "Universal C Runtime" out of the box (earlier you had to deploy a separate crt for each Visual Studio version).

Personally I'm also in favour of cutting out libc and invoking the windows API directly. But others might argue that using it as a shared abstraction that's very similar on Windows, Linux, OSX, BSD, is worth it.

And "native" is ill defined. It's just one more abstraction layer in the libc -> Win32-API -> NT-API -> Kernel chain. Libc isn't comparable to a java runtime or electron, which is what people usually mean when they talk about applications not being "native".

Re:Porting Rust's Std to Rustix by pjmlp on Tuesday January 04, 2022 @10:25AM (#29795720)
Kind of, it is still a language runtime specially on non-UNIX OSes.

The recent thread about cutting down the startup of C applications on Linux proves the point it isn't a zero cost library.

Re:Porting Rust's Std to Rustix by codeflo on Tuesday January 04, 2022 @08:53AM (#29794702)
I didn’t downvote, in fact, I’m a bit amused because the reason I used quotes is exactly because I’m not using any official definition! There’s no consensus definition of “native”, and I’m using “should” purely in the colloquial sense of “something I’d like to see”.