News for hackers. Stuff that matters.
Re:A four year plan for async Rust by withoutboats3 on Tuesday November 07, 2023 @11:53AM (#38179237)
What Rust libraries use async that you think should not? I can think of one, but otherwise every library I've encountered uses async because its intended for the kind of networking service that benefits from using non-blocking IO.
Re:A four year plan for async Rust by sapiogram on Tuesday November 07, 2023 @12:14PM (#38179619)
I had an absolutely awful experience when I tried using Sqlite through the sqlx crate. The work was mostly cpu-bound, but I didn't mind blocking the event loop for a few seconds here and there, so I thought I would just run it on Tokio's worker threads. Big mistake, I ended up getting extremely low CPU utilization due to a quirk (Or less charitably, a bug) in Tokio's scheduler.

I eventually rewrote the whole thing with rusqlite, but apart from being non-async, I found the API much less ergonomic to work with. But at least Rayon worked exactly as I expected.

Re:A four year plan for async Rust by couchand on Tuesday November 07, 2023 @11:55AM (#38179272)
Just because it's a network request doesn't mean it benefits from using async.

EDIT: to add a specific example, postgres. I understand from sfackler's perspective why it makes sense to maintain just an async library and a sync wrapper around it. But from the user's perspective there'a absolutely no reason that all of tokio should be required to talk to their db.

Re:A four year plan for async Rust by mplanchard on Tuesday November 07, 2023 @12:37PM (#38179984)
Nothing is stopping anyone from writing DB drivers that don't require async. If there's as much of a market for it as threads like this suggest, it seems like it would be a fairly popular project.
Re:A four year plan for async Rust by geodel on Tuesday November 07, 2023 @01:55PM (#38181157)
Huh, thats rather odd argument. It is like saying no one is stopping anyone to write new low level, memory-safe programing language, that won't have Rust style async system. If there is real market demand it could fairly popular project.

DB drivers, http servers, runtimes and so many other complex components are preferences of high skilled developers and not some objective market choices. Once its developed most application developers have to use it however user unfriendly it maybe.

Re:A four year plan for async Rust by mplanchard on Tuesday November 07, 2023 @07:05PM (#38184968)
One thing about Rust is that it's a fairly young language. We've had to implement a number of things ourselves that we wouldn't have had to implement in other languages.

If there's not enough community interest for there to already exist a solution for your problem, you really only have three options: write it yourself (including opening a PR to add it to an existing implementation), pay someone to write it, or switch languages. If none of these are options, you're just out of luck, and complaining about it is unlikely to do any good. I'm not saying you can't complain of course, but I am saying that you're more likely to get what you want via another path.

But regardless, I don't think it's an odd argument. Indeed, no one is stopping anyone from writing a new, low-level, memory-safe programming language! That's why Rust exists! Turns out the market was huge! Writing a new one now would be easier because of all the great ideas that Rust helped to prove work at scale. We're seeing this already with the success of languages like Zig, which makes different tradeoffs than Rust did and seems to be finding success in slightly different niches.

And sort of disproving your point, we use postgres, and I can think of three different implementations of postgres drivers offhand (sqlx, tokio-postgres, and diesel). I'll also note that the author of tokio-postgres also publishes https://docs.rs/postgres/latest/postgres/, which is not async! It's impressive how many options we have for such a complex thing in such a young ecosystem.

Re:A four year plan for async Rust by couchand on Tuesday November 07, 2023 @09:19PM (#38185961)
postgres is a wrapper around block-on(tokio-postgres).
Re:A four year plan for async Rust by withoutboats3 on Tuesday November 07, 2023 @12:07PM (#38179487)
As a user who has maintained network services that communicate with Postgres, I'm very glad that sfackler's postgres library doesn't perform blocking IO calls, as that would make it unfit for purpose for me.
Re:A four year plan for async Rust by couchand on Tuesday November 07, 2023 @12:28PM (#38179852)
Maybe you misunderstand me. I'm not saying there shouldn't be a non-blocking version. I'm saying async is not a silver bullet. Some programs/teams/environments have different constraints than you.

Would you also advocate for the kernel to deprecate blocking i/o?

Re:A four year plan for async Rust by withoutboats3 on Tuesday November 07, 2023 @01:11PM (#38180523)
Obviously, it'd be better for someone who wants to talk to postgres and doesn't need non-blocking IO if a version based on blocking IO existed. But what is your grand point? It's not some conspiracy that libraries for network services use async/await: they most often do it because their maintainers need to use async/await in the work that pays for them to provide this open source library free of charge to you.

And all you need to do to not deal with async is wrap it in one of the many different block_on implementations. Yes, that's less ideal than not having to pull in these dependencies; if avoiding those dependencies is mission critical for you, maybe yours could be the enterprise that pays for the blocking IO postgres library.

Re:A four year plan for async Rust by chlorion on Tuesday November 07, 2023 @01:42PM (#38180977)
"async" and threads are not the only options for concurrent IO.

I've never saw anyone discuss using poll(2) style concurrency here which is interesting but also kind of sad.

With the mio crate for example, it uses the operating systems "select/poll" interface. On Linix this is epoll(7), but other OSes have their own interface.

"poll" style concurrency is a lot less complicated for me. The general idea is to register IO handles such as sockets into a data structure and pass it into the kernel. The kernel will emit events whenever a handle is ready to do something. There is the option to go to sleep and block until one of the handles are ready but it's not required.

"async" is an abstraction on top of this. The async runtime is handling the event loop and other stuff for you which is very useful but also a little bit complicated.

I am not in any way against "async" style concurrency but I think a lot of libraries could avoid pulling in a runtime and still be concurrent.

Re:A four year plan for async Rust by insanitybit on Tuesday November 07, 2023 @01:56PM (#38181165)
"mio" was the game in town for years and it was definitely not ideal at all, it was promising but very low level and most people didn't use it if they could help it. People wanted async/await.
Re:A four year plan for async Rust by bryanlarsen on Tuesday November 07, 2023 @02:11PM (#38181355)
Several of my rust programs use tokio's select! macro as their main loop. I really like that pattern.
Re:A four year plan for async Rust by the__alchemist on Tuesday November 07, 2023 @11:57AM (#38179305)
> the kind of ... service that benefits from using non-blocking IO.

Async, in the sense from the article, and non-blocking aren't synonyms. Not using Async doesn't imply blocking.

Re:A four year plan for async Rust by withoutboats3 on Tuesday November 07, 2023 @12:00PM (#38179356)
What Rust libraries use non-blocking IO without using async syntax? I don't know of any. This interpretation of the request is new to me: I've exclusively heard complaints about libraries using async from people who want to use blocking IO.

(Also, in case it isn't obvious: I wrote the article in question.)

Re:A four year plan for async Rust by the__alchemist on Tuesday November 07, 2023 @12:05PM (#38179460)
I usually do non-blocking ops with hardware interrupts, DMA, distributed computing (over CAN etc), and multiple cores. For GPOS/Desktop PCs, threads, SIMD (GPU or CPU etc) are effective.

More generally than the concurrency operations I described, are any sort of event-loop or state machine, of which Async is one example.

Re:A four year plan for async Rust by dpc_01234 on Tuesday November 07, 2023 @01:23PM (#38180721)
Example from the recent past: I wanted a quick Rust code to stream s3 object, uncompress with zstd and untar the content to a directory. aws-sdk-s3 supports only async (tokio), tar crate only blocking, async-tar only async-std, async-compression only tokio. Hard to paper over it with `block_on` because of the streaming part. I don't remember what I actually did, but it doesn't matter - having to come up with a magic working combination is exactly the pain.

I just wish everything would at least still support blocking IO as a common denominator, so there's at least a baseline that is known to be possible. 99% of percent of the time I do not need benefits of async, because I write small software for relatively small, but still real use-cases. And using Rust ecosystem was easier just a few years ago than it is now for these use-cases, as blocking Rust was not yet relegated to be a second class citizen, replaced by an immature and fractured async.

Re:A four year plan for async Rust by insanitybit on Tuesday November 07, 2023 @01:44PM (#38181011)
> And using Rust ecosystem was easier just a few years ago than it is now for these use-cases

I used to write Rust web services (professionally) before async and it was definitely not easier for me. It was way harder. I'd end up with accidental hanging because some socket didn't have a timeout set on it properly, it was extremely leaky (why the hell am I talking to raw socket APIs just so that I can read from S3?), and it sucked compared to async. We've had radically different experiences, somehow.

Re:A four year plan for async Rust by dpc_01234 on Tuesday November 07, 2023 @05:24PM (#38183853)
> why the hell am I talking to raw socket APIs just so that I can read from S3

I don't know. https://doc.rust-lang.org/std/net/struct.TcpStream.html#meth... etc. were there for a while. Though I think that deadline-based timeouts would be way better to have. https://www.reddit.com/r/rust/comments/8b5krv/the_case_for_d... . Probably could be built around existing primitives. These things were never built/popularized because the community just jumped on async like some silver bullet.

I agree that building heavy duty networked services got easier with async, but at an expense of fracturing the ecosystem and dragging everything else into an MVP feature, which made things worse (at least in some respect) for other things. I personally don't do much web servers, and when I do I can just spawn lots of threads, terminate TLS with nginx anyway.

Again, I don't mind async on its own. I think it's great for what it is, and is useful when it's useful. But for decades tons of the web was built in Java, Python, RoR without async IO and it worked just fine. And I didn't have to play IO-type sudoku, and could expect that the most basic, native blocking IO is well supported, and not just an afterthought/wrapper.