News for hackers. Stuff that matters.
Re:How Rust Is Tilde’s Competitive Advantage [pdf] by notheguyouthink on Tuesday February 06, 2018 @01:52PM (#16318451)
> Using a language for a large project without some kind of async/await notation seems painful. How bad is it using only combinators for async (assuming you want to stay on stable)? And is there a date yet when async/await will be stabilized?

Can someone explain to me what async/await is, in the context of a language like Go? I've used Rust a fair amount, but Go has been my native language for ~5 years now.

With that said, aren't both languages synchronous languages where you can drop into another thread (or green thread) whenever you want?

What UX is improved by async/await?

Eg, in Go I never felt the need for async/await (as I've used in JavaScript at least). If anything, Go's is better than async/await to me, because if I call `foo := bar()`, I don't have to know if `bar()` is an asynchronous function.. it just works. Is there something missing there?

Again, I use Rust and Go similarly in this post because Go is my frame of reference. I thought they were the same on this front, though. No?

What's missing here? What am I missing here?

Re:How Rust Is Tilde’s Competitive Advantage [pdf] by steveklabnik on Tuesday February 06, 2018 @02:07PM (#16318599)
Yeah, you are missing some details. But this is a huge area, and it's hard to explain in a single HN comment. Basically, yes, the convenience of not needing to write async/await is useful, but in order to accomplish that, you need a bunch of other supporting decisions. These decisions make sense for Go, but do not make sense for Rust.

The first thing to say on this topic is that virtually all of this (including async/await) is a library in Rust, not a part of the language. You can build whatever model you want on top. The one I'm describing is the futures/tokio model, which is becoming the de-facto, default one. But if you want something else, you can do it! This already is a divergence from Go, where you have what the runtime gives you, and that's it.

Fundamentally, Go's model is cooperative coroutines, where the runtime does the pre-emption for you. (You can also call it yourself but as you note, this isn't usual.) It'll do this when you request something from the network, when you sleep, or when you use a channel.

Tokio's model, on the other hand, is single-threaded. Instead of a ton of gorutines, there's an event loop, and you place a chain of futures onto it. Each chain of futures can sort of be thought of as a goroutine, but it's also different. For example, Rust can know at compile time exactly how big each futures' stack is, and so can do a single, exactly correct allocation. Anyway, each future in the chain handles yielding for you.

Async/await, in Rust, is about making those chains easier to write.

Anyway, I hope that helps. Maybe someone else will have a good comparison too. There's a lot of options in this space, and similar terminology that means the same, but slightly different things, so it can be hard to get your head around.

Re:How Rust Is Tilde’s Competitive Advantage [pdf] by Rusky on Tuesday February 06, 2018 @02:27PM (#16318783)
Technically you can get the convenience of not needing to write async/await, with the same runtime implementation decisions underneath as Tokio/futures. Kotlin is an example of this approach- its coroutines are state machines but with implicit awaits.

This can even be extended to "awaiting across function calls" with effect polymorphism. I haven't seen this done in this context, but it would allow things like passing an async closure, or a generic type with an async trait implementation, to a normal function and having it automatically become an async function instead.

The real choice between explicit and implicit suspension points is thus syntactic, not implementation-driven. The reasoning there is more along the lines of "I like to see where my function might suspend" vs "I don't want to pepper my code with `await`."

Re:How Rust Is Tilde’s Competitive Advantage [pdf] by lmm on Tuesday February 06, 2018 @02:40PM (#16318883)
> I haven't seen this done in this context, but it would allow things like passing an async closure, or a generic type with an async trait implementation, to a normal function and having it automatically become an async function instead.

I've done that a fair bit in Scala using HKT. E.g. superclass is written in terms of a generic monadic type, one subclass implementation uses Future (actually EitherT with Future), another uses identity, a third uses Writer to track statistics...

Re:How Rust Is Tilde’s Competitive Advantage [pdf] by Rusky on Tuesday February 06, 2018 @06:30PM (#16320978)
Ah, right. I guess I have seen this sort of thing in Haskell then as well. When implemented with HKTs it usually gets really messy with things like monad transformers, so I prefer to think of asynchronicity more in terms of continuations.
Re:How Rust Is Tilde’s Competitive Advantage [pdf] by lmm on Wednesday February 07, 2018 @04:32AM (#16323265)
I find monad transformers much clearer than continuations; a monad transformer stack tells you exactly how your effects are going to be interleaved, which is necessarily complex, whereas a continuation could be doing anything at all, the way I see it.
Re:How Rust Is Tilde’s Competitive Advantage [pdf] by Rusky on Wednesday February 07, 2018 @06:12PM (#16328838)
No, continuations can be controlled similarly through effect polymorphism, which is what I was trying to describe.

The difference is not that continuations allow anything at all, but that your effects are commutative rather than layered like a monad transformer stack.

Re:How Rust Is Tilde’s Competitive Advantage [pdf] by lmm on Thursday February 08, 2018 @05:24AM (#16331285)
> continuations can be controlled similarly through effect polymorphism, which is what I was trying to describe.

Fair enough, but you're effectively talking about a secondary, novel type system, right? One of the things I like about monads is that they can be just plain old values of plain old datatypes.

> The difference is not that continuations allow anything at all, but that your effects are commutative rather than layered like a monad transformer stack.

Yeah, monad transformers do feel overly cumbersome for the cases where effects do commute. The trouble is that some effects don't commute, and I've yet to see a formalism that had a good solution to distinguishing effects that commute from those that don't. (I read a paper about a tensor product construct once, but couldn't really follow how it was supposed to work in practice)

Re:How Rust Is Tilde’s Competitive Advantage [pdf] by Rusky on Thursday February 08, 2018 @12:41PM (#16333937)
> you're effectively talking about a secondary, novel type system, right?

Yes, exactly.

The main reason I prefer language-level effects rather than monads is for composability with normal imperative control flow. Monads are written in terms of higher order functions passing closures around, and that gets really messy in a language where you not only have early return and loops with break and continue, but you also have destructors that need to run as you enter and leave scopes.

You can add that kind of stuff as monads but it gets really messy, and is basically completely untenable in a language like Rust that also cares about memory management. Even if Rust did have HKT, it would still be impossible to write a Monad trait that supports them all, for example.

This article makes some great points on the subject: http://blog.paralleluniverse.co/2015/08/07/scoped-continuati...

Re:How Rust Is Tilde’s Competitive Advantage [pdf] by steveklabnik on Tuesday February 06, 2018 @02:28PM (#16318791)
I have been meaning to dig into Kotlin's implementation, thanks for that! Is there any good reference documentation to dive into? Most of the stuff I saw was from a user's perspective.
Re:How Rust Is Tilde’s Competitive Advantage [pdf] by Rusky on Tuesday February 06, 2018 @06:28PM (#16320968)
Re:How Rust Is Tilde’s Competitive Advantage [pdf] by ronjouch on Tuesday February 06, 2018 @05:28PM (#16320540)
Adding a recent detailed take on the situation, by another Rustlang contributor: https://manishearth.github.io/blog/2018/01/10/whats-tokio-an... . Hope that helps :)
Re:How Rust Is Tilde’s Competitive Advantage [pdf] by sagichmal on Tuesday February 06, 2018 @02:00PM (#16318525)
> Can someone explain to me what async/await is, in the context of a language like Go?

As far as I know:

    c := make(chan int)
    go func() { c <- 1 }() // async
    i := <-c               // await
Re:How Rust Is Tilde’s Competitive Advantage [pdf] by ehsanu1 on Tuesday February 06, 2018 @02:04PM (#16318564)
Goroutines allow you to write async code ergonomically. Without those, you'd have to write callbacks, which is very unergonomic.

Here's a before and after diff of some prototype async/await usage in Rust: https://github.com/mehcode/shio-rs/commit/8078a34c075bf2f52c...

Re:How Rust Is Tilde’s Competitive Advantage [pdf] by lmm on Tuesday February 06, 2018 @02:47PM (#16318949)
async/await means you mark all your yield points explicitly; rather than having the runtime implicitly preempt you whenever it chooses, you mark the points at which task switching can happen, and at every other point it's impossible (equivalently it's as though any block of code that doesn't contain an "await" were a critical section). The syntax strikes a nice balance, making these markers as lightweight as possible but no lighter: you don't want the yield points to be completely invisible, but you don't want them to distract too much from reading through the straight-through happy-path control flow.

I find that having one implicit, pervasive, uncontrolled effect in a given piece of code is ok, but having multiple uncontrolled effects is very much not ok, because the ways those effects will interact can be very surprising. E.g. https://glyph.twistedmatrix.com/2014/02/unyielding.html gives an argument for using explicit async/await rather than go-style implicit (green) threads, in terms of how task switching will interact with state mutation. You can make similar arguments in terms of how task switching interacts with error handling or logging or database transactions or... - one uncontrolled effect is fine, multiple uncontrolled effects cause chaos when they interact. Unfortunately the kind of examples where one can see multiple effects in a single application tend to be quite big by their very nature (and if your app is small enough to only need to have one kind of effect, then using a language in which that particular effect is uncontrolled is probably fine, perhaps even a good idea).

Re:How Rust Is Tilde’s Competitive Advantage [pdf] by kbenson on Wednesday February 07, 2018 @12:35AM (#16322431)
> async/await means you mark all your yield points explicitly; rather than having the runtime implicitly preempt you whenever it chooses, you mark the points at which task switching can happen, and at every other point it's impossible (equivalently it's as though any block of code that doesn't contain an "await" were a critical section).

That's not how I understood it at all. I always assumed that it just promoted explicitly acknowledging when the effect of an async block would be present, but that the normal yield points (traditionally syscalls and IO requests in OS threads) generally still held for the actual execution of the other code.

I mean, if we aren't allowing separate code paths to execute (and thus reducing wait on resources such as IO), what's the point?

Re:How Rust Is Tilde’s Competitive Advantage [pdf] by lmm on Wednesday February 07, 2018 @04:30AM (#16323256)
> That's not how I understood it at all. I always assumed that it just promoted explicitly acknowledging when the effect of an async block would be present, but that the normal yield points (traditionally syscalls and IO requests in OS threads) generally still held for the actual execution of the other code.

Put it this way: the language implementation won't preempt you except at your explicit yield points (some standard library functions for things like I/O will and should be yield points that you have to call with async/await). If the language implementation happens to be running as a userspace process on a preemptive multitasking OS then it will still be subject to the same preemption rules as any other userspace process on that OS, but if anything this is usually counterproductive (e.g. priority inversions are almost guaranteed); recommended practice when working in green-thread-based systems is to run with one thread per CPU core and maybe even pin threads to cores if the OS lets you do that, because the language's own M:N scheduling will keep those threads fully occupied and OS scheduling is only going to get in the way.

Re:How Rust Is Tilde’s Competitive Advantage [pdf] by kbenson on Wednesday February 07, 2018 @10:15AM (#16324901)
I think I see what's going on here. You're answering "in the context of Go" (which is correct, and what was corrected) and I'm interpreting it as a general statement about what async/await mean as general concepts (i.e. in other languages as well). My mistake, I wasn't paying close enough attention. :)
Re:How Rust Is Tilde’s Competitive Advantage [pdf] by kbenson on Wednesday February 07, 2018 @06:18PM (#16328871)
s/and what was corrected/and what was requested/ Ugh, so many typos or word mistakes lately. :/
Re:How Rust Is Tilde’s Competitive Advantage [pdf] by dilap on Tuesday February 06, 2018 @11:49PM (#16322307)
Having used both, I'd be pretty shocked if anyone found Twisted to be easier to reason about or less prone to bugs than Go...
Re:How Rust Is Tilde’s Competitive Advantage [pdf] by laurencerowe on Wednesday February 07, 2018 @12:54AM (#16322498)
Twisted predates async/await syntax in Python.
Re:How Rust Is Tilde’s Competitive Advantage [pdf] by ddorian43 on Monday February 12, 2018 @07:46AM (#16358239)
Isn't gevent still better than async/await ? (if it had core support it would be, kinda like golang)
Re:How Rust Is Tilde’s Competitive Advantage [pdf] by chatmasta on Tuesday February 06, 2018 @01:58PM (#16318513)
Async/await makes sense in python or JavaScript because it’s syntactical sugar that improves readability/maintainability and mitigates against concurrency errors stemming from the event loop (JS) or the GIL (python). I haven’t worked with Go beyond playing with it a bit, but it seems async/await would be utterly useless in Go since goroutines already accomplish the same thing in a much more performant manner.
Re:How Rust Is Tilde’s Competitive Advantage [pdf] by eropple on Tuesday February 06, 2018 @03:12PM (#16319169)
C# was, as far as I'm aware, the first major language to implement async/await atop of Task<T> (akin to Java's Future<T>). The CLR uses native threads.

Go channels are also orthogonal to async/await. Message passing is not a substitute for futures/tasks, though it can be used to achieve similar goals. I would be extremely cautious about claiming that Go channels would be "more performant" than an otherwise-equivalent futures implementation, too.

Re:How Rust Is Tilde’s Competitive Advantage [pdf] by wahern on Tuesday February 06, 2018 @06:02PM (#16320769)
Goroutines are not the same thing as Go channels.
Re:How Rust Is Tilde’s Competitive Advantage [pdf] by eropple on Tuesday February 06, 2018 @06:27PM (#16320959)
Sorry, yes. Fibers (Go didn't invent them) also aren't directly equivalent because they don't allow explicit yields and composition.
Re:How Rust Is Tilde’s Competitive Advantage [pdf] by wahern on Tuesday February 06, 2018 @08:12PM (#16321512)
Fibers don't, but goroutines + channels + closures do. They permit composition using the same call/return syntax and semantics as normal function calls.

Futures and promises don't. Async/await is closer, but only by creating a second class of function incompatible with normal functions.

Re:How Rust Is Tilde’s Competitive Advantage [pdf] by jabl on Tuesday February 06, 2018 @02:38PM (#16318869)
> Again, I use Rust and Go similarly in this post because Go is my frame of reference. I thought they were the same on this front, though. No?

No. Rust used to have a green threads runtime like Go (and Erlang/BEAM, Haskell, etc.) but that was removed before the 1.0 release. So today, a thread in Rust is a heavyweight OS thread, like in C/Java and most mainstream languages.

Re:How Rust Is Tilde’s Competitive Advantage [pdf] by tytytytytytytyt on Tuesday February 06, 2018 @08:02PM (#16321461)
The intro Rust book says you can easily find crates for green threads of you really want them. Is this not the case?
Re:How Rust Is Tilde’s Competitive Advantage [pdf] by tiuPapa on Wednesday February 07, 2018 @12:12AM (#16322373)
I am not entirely sure, it appears like due to the present status of std in rust, any green threads lib is prone to Undefined Behaviour whenever something uses Thread Local Storage.
Re:How Rust Is Tilde’s Competitive Advantage [pdf] by notheguyouthink on Wednesday February 07, 2018 @02:18PM (#16327180)
Yea, I'm aware - but you can spin off a async execution and then send data back over channels (I forget what Rust calls), as well as wrap that whole thing up in a function with a return value so that the caller has no idea of the threaded execution taking place. Right?

Sure the implementation of the threading differs greatly, but I was mainly referring to that I can spawn async behavior, and use synchronous blocking methods to get data back from that async behavior - AND encapsulate it.

All of these are very different than, say, what JavaScript goes through anytime async is involved.

Re:How Rust Is Tilde’s Competitive Advantage [pdf] by steveklabnik on Wednesday February 07, 2018 @04:10PM (#16328028)
Rust also has channels.