News for hackers. Stuff that matters.
Re:Async-await on stable Rust by takeda on Thursday November 07, 2019 @09:30AM (#21473464)
Why would it be different for async code than sync code? The goal of Rust's checker is to track lifetime of an object so for example it knows that at the end of a function the object should be freed. Async shouldn't matter here.
Re:Async-await on stable Rust by GolDDranks on Thursday November 07, 2019 @09:39AM (#21473547)
The point is that Rust's borrow checker can't reason about lifetimes very well over function boundaries. It can reason about coarse things that are expressable in the type language, but everything more nuanced than that, such as reasoning about how control flow affects the lifetimes is limited to inside function bodies.

The difference between synchronous code and async code implemented as libraries is that async code involves jumping in and out of functions a lot, while employing runtime library code in between. A piece of code that is conceptually straightforward, may, in the async case, involve multiple returns and restores. In the sync case it doesn't need to do that, since it just blocks the thread and does the processing in other threads and in kernel land.

Rust's async/await support makes it possible to write code that is structurally "straightforward" in a similar way than synchronous code would be. That allows the borrow checker to reason about it in a similar way it would reason about sync code.

Re:Async-await on stable Rust by dunkelheit on Thursday November 07, 2019 @09:56AM (#21473697)
> The point is that Rust's borrow checker can't reason about lifetimes very well over function boundaries. It can reason about coarse things that are expressable in the type language, but everything more nuanced than that, such as reasoning about how control flow affects the lifetimes is limited to inside function bodies.

BTW this is a big pain point for me (unrelated to async). Code like this:

  let ref = &mut self.field;
  self.helper_mutating_another_field();
  do_something(ref);
gets rejected because self.helper_mutating_another_field() will mutably borrow the whole struct. The workaround is either to inline the helper or factor out a smaller substruct so that helper can borrow that which doesn't always look good.

Of course it is preferable that all information needed for the caller to check if the call is correct is contained in the function signature but it truly is frustrating to see the function body right there, know that it doesn't violate borrowing rules and still get the code calling it rejected.

Re:Async-await on stable Rust by entropicdrifter on Thursday November 07, 2019 @10:30AM (#21474032)
Couldn't you just pass in the (other) borrowed field as an argument for the function? If you need it to work without adding the argument when called outside of the class, you could overload it with a version that borrows the field and passes it to the version that takes the field as an argument, right?

I'm newish to Rust, so this is just an intuitive guess. Please let me know if I'm wrong.

Re:Async-await on stable Rust by dunkelheit on Thursday November 07, 2019 @12:15PM (#21475053)
Yes, it works, but feels unnatural. I also don't like the possibility (quite remote, I admit) that the function gets called with a field from another instance.
Re:Async-await on stable Rust by esotericn on Thursday November 07, 2019 @01:25PM (#21475812)
The latter case should be impossible if the method is private?
Re:Async-await on stable Rust by saghm on Thursday November 07, 2019 @09:49PM (#21479601)
In Rust, privacy isn't enforced like that. Private just means that things outside the module can't access them. There's no concept of privacy at the instance level.
Re:Async-await on stable Rust by esotericn on Friday November 08, 2019 @05:37AM (#21481157)
Yes. So don't make any methods that could potentially be passed incorrect arguments public.
Re:Async-await on stable Rust by saghm on Friday November 08, 2019 @11:46AM (#21483974)
That still doesn't preclude the possibility that within the module the function gets called with a field from another instance. I think the idea is that by making it a method that just takes a reference to self, it's impossible to accidentally mutate a field on a different instance, while taking a reference to the field itself doesn't prevent the programmer from accidentally calling it with the wrong instance of the field.
Re:Async-await on stable Rust by Rusky on Thursday November 07, 2019 @11:15AM (#21474498)
Yes, that's the usual workaround.
Re:Async-await on stable Rust by rom1v on Thursday November 07, 2019 @05:51PM (#21478135)
Yes, encapsulation sometimes conflicts with the borrowing rules. After all, this is not so surprising: enforcing the borrowing rules requires to follow every concrete access to resources, while encapsulation aims to abstract them away.
Re:Async-await on stable Rust by steveklabnik on Thursday November 07, 2019 @10:09AM (#21473824)
It's really tricky; there's a tension here between just making your code work, and not making it brittle to further modification.
Re:Async-await on stable Rust by asdkhadsj on Thursday November 07, 2019 @09:48AM (#21473627)
If you're familiar with this, can you describe some of those new concepts in ... slightly more detail? I say slightly, because I'm still seeking high level explanations, but at the same time I'm curious what new features might be making this async lifetime talk possible.

To further frame that question: I had assumed Rust was implementing Async within the capabilities of normal Rust. Such that, if lifetimes were being managed across function bounds, I assumed the lifetime would have "simply" been bound to the scope of the executor, polling the future you're actively waiting on.

However quickly I can see confusions in that description. Normal lifetimes within a function would need to bubble up and be managed by a parent, since that function's lifetime is, as you put it, being jumped in and out of frequently.

So I imagine that is partly where new features come into play? Giving Rust the ability to take a normal lifetime and extend or manage it in new ways?

Re:Async-await on stable Rust by GolDDranks on Thursday November 07, 2019 @09:40PM (#21479560)
I'm not going to elaborate super deeply in a HN comment, but here's the gist: inside a function body, you can take a reference to a thing, and store that reference in a variable. Then if that function were "yields" to the executor, we have a situation that we couldn't have with sync code: in sync code, the "yielding" only happens at the function return, and by that point, all the internal references must be gone, as the function stack frame is going to disappear. But with async, as the yielding can happen without the function actually ending, we have to be able to store the stack frame, and the references stay alive. The new feature helps the compiler to reason about this. Before the "yields" were implemented as just returns, so the compiler didn't allow borrowing over yield points.
Re:Async-await on stable Rust by asdkhadsj on Saturday November 09, 2019 @09:48AM (#21491815)
Ah hah, that sounds awesome, thanks!

Yea my past experience with Futures in Rust, mostly manually, made them feel like they were "No magic, just Rust" and so the Yields were just a lot of returns.

I had thought the .await impl was largely just standardization around the syntaxes. Something you could do manually, so to say.

The yielding holding the stack frame, in my head then, sounds quite similar to actually returning. However, it's a special return that stashes that stack somewhere for later use. Neat!

Re:Async-await on stable Rust by epage on Thursday November 07, 2019 @09:48AM (#21473618)
The challenge with async code is that the state across yield points is put into a struct and if a reference to a stack variable is used across yield points, then that struct is self-referential which Rust can't reason about (yet?). I honestly do not know who much can be recreated with `unsafe` and `Pin` vs how much is built-in.

I'd love for Rust to eventually get a Move trait (something like C=+ move constructors) to resolve this. Besides some complexity in designing it, there is resistance from some corners about having anything execute on a move.

Re:Async-await on stable Rust by stbuehler on Thursday November 07, 2019 @12:34PM (#21475231)
That is exactly what `async` blocks are about: they do support self-referential structs, and `Pin` is what allows us to use those in a safe way, as `Pin`ned data can't be moved again (unless the data is `Unpin`, which self-referential structs are not), so the self-references are safe.
Re:Async-await on stable Rust by aliceryhl on Thursday November 07, 2019 @09:35AM (#21473523)
In Rust, it is normally not possible or at least very difficult to create structs where one field references another, and if you were to create a future that borrows some field, awaits a future, and uses the borrowed field, the resulting future will need to have a field with a reference to another.

This is the challenge and async await lets you make this kind of self referential types without unsafe code.

Re:Async-await on stable Rust by antisemiotic on Thursday November 07, 2019 @09:57AM (#21473718)
That's just the "Pin" type, which is heavily used in async code behind (and occasionally in front) the scenes, but is by no means restricted to it.
Re:Async-await on stable Rust by Nemo157 on Thursday November 07, 2019 @10:18AM (#21473911)
It's impossible to use the guarantees provided by the `Pin` type without `unsafe` code, except in `async fn`.