News for hackers. Stuff that matters.
Re:You might want async in your project by dpc_01234 on Saturday September 09, 2023 @04:20PM (#37449872)
I might need a lot of stuff in my software. Eventually. I might need distributed database, or to to scale it out to run on multiple machines, and then maybe Raft, or reactive architecture, zero-copy IO, or incremental updates or ... or ... the list goes on and on.

Thinking too much and in particularly going with over complicated solutions from the very start because "might" is just bad engineering.

Also, even if I do need async in a certain place, doesn't mean I need to endure the limitations and complexity of async Rust everywhere in my codebase. I can just spawn a single executor and pass around messages over channels to do what requires async in async runtime, and what doesn't in normal and simpler (and better) blocking IO Rust.

You need async IO? Great. I also need it sometimes. But that doesn't explain the fact that every single thing in Rust ecosystem nowadays is async-only, or at best blocking wrapper over async-only. Because "async is web-scale, and blocking is not web-scale".

Edit: Also the "just use smol" comically misses the problem. Yeah, smol might be simpler to use than tokio (it is, I like it better personally), but most stuff is based on tokio. It's an uphill battle for the same reasons using blocking IO Rust is becoming an uphill battle. Only thing better than using async when you don't want to is having to use 3 flavors (executors) of async, when you didn't want to use any in the first place.

Everything would be perfect and no one would complain about async all the time if the community defaulted to blocking, interoperable Rust, and then projects would pull in async in that few places that do actually need async. But nobody wants to write a library that isn't "web-scale" anymore, so tough luck.

Re:You might want async in your project by the__alchemist on Saturday September 09, 2023 @04:22PM (#37449894)
I also see the `Async` vs `blocking` false dichotomy in embedded rust discussions. `Async/Await` != asynchronous execution.
Re:You might want async in your project by vc8f6vVV on Sunday September 10, 2023 @01:36PM (#37458081)
In some sense it is. Async is a glorified future, and future is a glorified thread management, and threads are a way to facilitate asynchronous execution. You can also create a threadless runtime, but then you are relying on OS threads (e.g. I/O or XHR), otherwise you are simply combining function calls (for which we already have language syntax).
Re:You might want async in your project by Ar-Curunir on Saturday September 09, 2023 @04:45PM (#37450102)
> I also need it sometimes. But that doesn't explain the fact that every single thing in Rust ecosystem nowadays is async-only, or at best blocking wrapper over async-only.

Now that’s just plainly untrue.

Re:You might want async in your project by dpc_01234 on Saturday September 09, 2023 @05:01PM (#37450244)
Yes, yes. You're right. Though it does sometimes feel like it.
Re:You might want async in your project by karmakaze on Saturday September 09, 2023 @08:49PM (#37451836)
I guess one thing going for it is that then Rust is a lot more like Node, which could help increase its popularity. I wonder if Rust at some point in its history became self-serving (bigger == better).
Re:You might want async in your project by insanitybit on Saturday September 09, 2023 @08:57PM (#37451885)
> But nobody wants to write a library that isn't "web-scale" anymore, so tough luck.

It's more like "I want to be able to put timeouts in my code". 99% of why I want async is so that if something takes too long I can just stop that. That is incredibly hard to do without async.

Re:You might want async in your project by vc8f6vVV on Sunday September 10, 2023 @01:45PM (#37458159)
Not sure about Rust, but in other languages that don't have async: create a queue, spawn a thread with your task, thread with sleep and wait for a message from any of those two. Kill the still running thread when you get the message. Can't say it's incredibly hard (unless it's Javascript or you work in a single-threaded model in general).
Re:You might want async in your project by insanitybit on Sunday September 10, 2023 @02:24PM (#37458550)
> Kill the still running thread when you get the message

This is extremely difficult. I mentioned elsewhere that the only way to kill a thread is through the pthread_cancel API, which is very dangerous.

Languages with larger runtimes can get around this because they have all sorts of things like global locks or implicit cooperative yielding. So they don't ever have to "kill" a thread, they actually just join it when it yields.

Re:You might want async in your project by vc8f6vVV on Sunday September 10, 2023 @03:11PM (#37458959)
As I understand when you are blocked on I/O and sends a signal to the waiting thread, that system call will simply be released and return an error. Ruby (Java etc.) does make it simple because of GC, so I don't need to worry about file descriptor leaks etc. But talking about Rust, shouldn't it be a part of a thread management? Basically if an error happens during normal blocking system call, it goes through the same sequence, no? E.g. you have to release any thread-local allocations no matter by each way system call was terminated. Rust threads are supposed to be memory safe, not sure about file descriptors. I don't quite understand what you mean by "yielding" though.
Re:You might want async in your project by klabb3 on Sunday September 10, 2023 @02:19PM (#37458492)
Genuinely curious: looks to me like you can set timeout on sockets (TcpStream) with std. Does that not work? What other kinds of timeouts do you typically need?
Re:You might want async in your project by insanitybit on Sunday September 10, 2023 @02:22PM (#37458534)
Relying on the socket APIs is a bit painful. For one thing, what if that socket is shared across tasks? What if I want to do per-request timeouts? What if I don't want to expose the socket APIs to callers?

What if my work isn't related to socket timeouts? For example, downloading a multi-part file? I may want a very long socket timeout but still have a distinct timeout for the individual chunked operations. It might take 30 seconds to grab an entire file but if one chunk takes >3 seconds I may want to time out.

What if my work involves no IO at all?

The ability to cancel work in progress is extremely important to me.

Re:You might want async in your project by klabb3 on Sunday September 10, 2023 @03:04PM (#37458903)
> For one thing, what if that socket is shared across tasks?

Yeah that’s an issue. In Go they sync it (thread safe), which in Rust would translate to interior mutability.

> What if I want to do per-request timeouts?

Ah you’re right in http2 there can be multiple concurrent reqs per conn. Go still allows request based timeouts, but I wonder if that’s possible with the limited primitives in std. It’s also true that this is a case where the inner conn should not be exposed.

> I may want a very long socket timeout but still have a distinct timeout for the individual chunked operations.

Right! That’s typically done by extending the deadline for every chunk. Ie the user/caller needs a way to set timeouts.

> The ability to cancel work in progress is extremely important to me.

Yes for sure. I was just curious. Btw which libs are you referring to for network requests? I’d like to see their APIs.

Re:You might want async in your project by insanitybit on Tuesday September 12, 2023 @01:47PM (#37484851)
> Btw which libs are you referring to for network requests?

Consider this: https://docs.rs/rusoto_s3/latest/rusoto_s3/trait.S3.html

You're given a trait and, understandably, this trait does not expose any sockets to you (it may not even be backed by sockets).

Re:You might want async in your project by Tigress8780 on Saturday September 09, 2023 @09:53PM (#37452183)
IO is not a part of the async runtime contract (I don't know if this is good or bad), and Tokio & futures famously have different `Async{Read,Write}` traits. I once had to do this [0] to adapt between them.

This means that any crate that uses IO will be bound to a limited number of Runtimes. Everything being Tokio-only is pretty bad (though Tokio itself is great), but here we are...

[0] https://github.com/bluejekyll/trust-dns/pull/1373#issuecomme...