Re:\Device\Afd, or, the Deal with the Devil that makes async...
by yccs27 on Monday March 13, 2023 @04:40PM
(#35142221)
I love it when hacky programming is compared to witchcraft, because it describes the feeling of reading and writing it.
Reminds me of the "technical interview" [0] from yesterday‘s front page.
Re:\Device\Afd, or, the Deal with the Devil that makes async...
by doublerabbit on Monday March 13, 2023 @05:03PM
(#35142593)
I can name a few through my joirneys.
Perl is a temple. You construct some ritualistic temple of a project, leave it be for years and will stand happily without any issues. Come back many months later, add a single comment; total Indiana Jones scenario.
TCL is voodoo, it does stuff that shouldn't work but does awesomely. How it does it is just voodoo.
Ruby is just magic. You create a thing, change one thing and the next thing you know you've suddenly got two features for the price of one.
Re:\Device\Afd, or, the Deal with the Devil that makes async...
by tracker1 on Monday March 13, 2023 @05:12PM
(#35142749)
PHP is a sheep with one droopy eye and one of the ears actually looks like it belongs to a donkey, but the fur/wool is at first glance, well-conditioned.
Re:\Device\Afd, or, the Deal with the Devil that makes async...
by dblohm7 on Monday March 13, 2023 @06:51PM
(#35144264)
Good god. I like nothing about this piece. I hate the claim that Windows networking is "duct-taped" in. I hate that they're using undocumented stuff. I hate that they're trying to shoehorn Windows into a Unix-y model; in my experience this is often symptomatic of failing to consider all platforms when originally designing the higher-level abstractions. Just blah.
> To make a long story that I don’t fully understand short, you need to have buffers that are pinned in memory and aren’t touched as long as the operation is in progress.
I don't understand why this is so confusing.
> I doubt that Microsoft is really rushing to make it so that you can’t run Javascript on Windows anymore.
Nah, but don't be surprised if suddenly a bunch of apps receive compatibility shims.
Re:\Device\Afd, or, the Deal with the Devil that makes async...
by pornel on Monday March 13, 2023 @07:23PM
(#35144693)
Rust sometimes is very Linux-centric. This problem has been considered when async was in the draft stage, and shrugged off, because it's just Windows. macOS in Rust is also treated as "it's just a weird Linux, right?".
Re:\Device\Afd, or, the Deal with the Devil that makes async...
by dblohm7 on Tuesday March 14, 2023 @02:07PM
(#35155835)
That's how things have always been done at Mozilla (I spent several years there dealing with Windows-specific stuff for Firefox)... I kinda wonder if Rust inherited a lot of that attitude from there.
Re:\Device\Afd, or, the Deal with the Devil that makes async...
by Dylan16807 on Tuesday March 14, 2023 @02:22PM
(#35156139)
My understanding is the API they want notifies when data is ready, then you make a buffer and fill it immediately. In that situation a significantly smaller set of errors is possible and you don't have to figure out disposal of a pinned and possibly-in-use buffer if you decide to cancel.
Re:\Device\Afd, or, the Deal with the Devil that makes async...
by fxtentacle on Tuesday March 14, 2023 @12:45AM
(#35147553)
I belive the issue here is that the author very much likes Rust and is not willing to openly admit that Rust's needs Windows-specific features.
I found it odd that even the examples for low-level C APIs were presented through Rust wrappers.
"the overarching idea is that IOCP doesn’t gel well with not only the Future model, but also with Rust’s borrow checker as a whole" comes pretty close, though, but it's still lacking the insight that it's an issue in Rust's language design and not in Windows.
Also, I've heard a lot of praise about Microsoft's networking integration which made things like mounting network drives possible and reliable as early as Windows 2000. But of course, if you refuse to use the recommended APIs, you'll suffer for it.
I wonder if the author would describe it in similar detail, if they open a Linux device like /dev/video0 and send a V4L syscall to it. Because that's the equivalent to using the Afd device.
That said, I enjoyed their word choices and writing style.
Re:\Device\Afd, or, the Deal with the Devil that makes async...
by dblohm7 on Tuesday March 14, 2023 @02:08PM
(#35155855)
> Also, I've heard a lot of praise about Microsoft's networking integration which made things like mounting network drives possible and reliable as early as Windows 2000. But of course, if you refuse to use the recommended APIs, you'll suffer for it.
Re:\Device\Afd, or, the Deal with the Devil that makes async...
by tiraniddo on Tuesday March 14, 2023 @07:41AM
(#35149961)
The article also felt like Rust does it this way because all these other projects which came before it did it this way. Someone originated the technique and no one questions its validity, without seeing if there was an alternative method to achieve the same result.
I did spend about 20 minutes looking into this, it seems that there's an (admittedly undocumented) IOCTL which is in the Winsock headers (MSWsock.h) called SIO_EXT_POLL which seems to just use the poll device IO control code and so they could probably do this without messing with low-level AFD weirdness.
That said you'd think with Microsoft going all in on Rust, last I heard at least, that they could find a way to re-implement this code correctly. Hell, if they at least documented the IOCTL they could probably get co-pilot to do it for them ;)
Re:\Device\Afd, or, the Deal with the Devil that makes async...
by dblohm7 on Tuesday March 14, 2023 @02:12PM
(#35155950)
> The article also felt like Rust does it this way because all these other projects which came before it did it this way. Someone originated the technique and no one questions its validity, without seeing if there was an alternative method to achieve the same result.
Precisely. As somebody with extensive experience porting cross-platform applications to run on Windows, it smells more to me like "everybody involved in this is actually quite ignorant about Windows," instead of (what I am assuming is the intended) "look at this crazy shit we had to do because Windows sucks."
Re:\Device\Afd, or, the Deal with the Devil that makes async...
by anaisbetts on Monday March 13, 2023 @07:50PM
(#35145036)
Windows does offer an official API for this, as the article states. Unfortunately, everyone wants an API that works exactly like epoll, and IOCP just Doesn't Work Like That. It's actually way closer to how most people would expect async I/O to work, were they to design it from scratch without the well being poisoned by knowledge of epoll.
However, every *nix / macOS person writes their epoll code, then finds out that it Doesn't Work That Way on Windows, so this gives them back their epoll-like API.
Re:\Device\Afd, or, the Deal with the Devil that makes async...
by zdimension on Monday March 13, 2023 @09:25PM
(#35146058)
From a Windows development point of view, Linux is way behind. Async I/O has been at the core of the NT kernel since the very beginning (since VMS, even), whereas on Linux (and Unix) everything has been designed with blocking I/O in mind. IOCP match the way people usually think about asynchronous processing, but people have just become used to the weird way poll and epoll do things. It's like saying Windows is behind Linux because it doesn't support Unix permissions.
Re:\Device\Afd, or, the Deal with the Devil that makes async...
by anaisbetts on Monday March 13, 2023 @07:44PM
(#35144975)
A story I heard while working on Kernel and Drivers at MS, is that this acronym originally stood for "Another Fucking Driver", named by Dave Cutler once a kernel engineer told him that it was necessary and asked him for a good name. Obviously before they shipped, they needed to come up with a more Palatable name, so "Auxiliary function driver" was the name they came up with.
[0] https://news.ycombinator.com/item?id=35120084
Perl is a temple. You construct some ritualistic temple of a project, leave it be for years and will stand happily without any issues. Come back many months later, add a single comment; total Indiana Jones scenario.
TCL is voodoo, it does stuff that shouldn't work but does awesomely. How it does it is just voodoo.
Ruby is just magic. You create a thing, change one thing and the next thing you know you've suddenly got two features for the price of one.
PHP is a sheep.
> To make a long story that I don’t fully understand short, you need to have buffers that are pinned in memory and aren’t touched as long as the operation is in progress.
I don't understand why this is so confusing.
> I doubt that Microsoft is really rushing to make it so that you can’t run Javascript on Windows anymore.
Nah, but don't be surprised if suddenly a bunch of apps receive compatibility shims.
Pinning a buffer is easy. Dealing with ownership ramifications in all situations is confusing.
I found it odd that even the examples for low-level C APIs were presented through Rust wrappers.
"the overarching idea is that IOCP doesn’t gel well with not only the Future model, but also with Rust’s borrow checker as a whole" comes pretty close, though, but it's still lacking the insight that it's an issue in Rust's language design and not in Windows.
Also, I've heard a lot of praise about Microsoft's networking integration which made things like mounting network drives possible and reliable as early as Windows 2000. But of course, if you refuse to use the recommended APIs, you'll suffer for it.
I wonder if the author would describe it in similar detail, if they open a Linux device like /dev/video0 and send a V4L syscall to it. Because that's the equivalent to using the Afd device.
That said, I enjoyed their word choices and writing style.
Yup, you nailed it!
I did spend about 20 minutes looking into this, it seems that there's an (admittedly undocumented) IOCTL which is in the Winsock headers (MSWsock.h) called SIO_EXT_POLL which seems to just use the poll device IO control code and so they could probably do this without messing with low-level AFD weirdness.
That said you'd think with Microsoft going all in on Rust, last I heard at least, that they could find a way to re-implement this code correctly. Hell, if they at least documented the IOCTL they could probably get co-pilot to do it for them ;)
Precisely. As somebody with extensive experience porting cross-platform applications to run on Windows, it smells more to me like "everybody involved in this is actually quite ignorant about Windows," instead of (what I am assuming is the intended) "look at this crazy shit we had to do because Windows sucks."
However, every *nix / macOS person writes their epoll code, then finds out that it Doesn't Work That Way on Windows, so this gives them back their epoll-like API.