News for hackers. Stuff that matters.
Re:Async-await on stable Rust by jstimpfle on Thursday November 07, 2019 @12:00PM (#21474923)
> I believe will make rust pretty much the easiest low-level platform to develop for.

This stuff is easily available for C. It's just a function call instead of syntactic sugar. On Windows, use Fibers. On Linux, there is a (deprecated) API as well (forget the name). Or use a portable wrapper library.

Or just write a little assembly. Can't be hard.

Re:Async-await on stable Rust by ComputerGuru on Friday November 08, 2019 @12:32AM (#21480181)
How exactly do I write embedded code for a tiny chip on a different architecture with just 4K of RAM and no OS that uses the Win32 fibers?
Re:Async-await on stable Rust by jstimpfle on Friday November 08, 2019 @12:40AM (#21480195)
Probably not using green threads / async at all? That would require a number of stacks as well as a little runtime, which will quickly eat up your 4K.
Re:Async-await on stable Rust by tmandry on Friday November 08, 2019 @02:39AM (#21480559)
Just to clarify for those following along, Rust async code does not use green threads and doesn't require a stack per task.
Re:Async-await on stable Rust by jstimpfle on Friday November 08, 2019 @04:58AM (#21481034)
I'm missing some of the technical details here, but from a quick glance of the article it seems like Rust's futures are lazy. I.e. a stack would only be allocated when the future is actually awaited. But in order to execute the relevant code, a call stack per not-finished future is still needed, or am I missing something?
Re:Async-await on stable Rust by dthul on Friday November 08, 2019 @05:34AM (#21481144)
Afaik Rust's futures compile to a state machine, which is basically just a struct that contains the current state flag and the variables that need to be kept across yield points. An executor owns a list of such structs/futures and executes them however it sees fit (single-threaded, multi-threaded, ...). So there is no stack per future. The number of stacks depends on how many threads the executor runs in parallel.
Re:Async-await on stable Rust by jstimpfle on Friday November 08, 2019 @07:20AM (#21481582)
> the current state flag the variables that need to be kept across yield points.

you mean like... a stack frame?

Re:Async-await on stable Rust by littlestymaar on Friday November 08, 2019 @09:38AM (#21482515)
Like a stack frame, but allocated once of a fixed size, instead of being LIFO allocated in a reserved region (which itself must be allocated upfront, when you don't know how big you're going to end up).

The difference being: if your tasks need 192B of memory each, and you spawn 10 of each, you just consumed a little less than 2kB. With green threads, you have 10 times the starting size of your stack (generally a few kB). That makes a big difference if you don't have much memory.

Re:Async-await on stable Rust by jstimpfle on Friday November 08, 2019 @11:33AM (#21483813)
So that's actually green threads in my book (in a good implementation I expect to be able to configure the stack size), with the nice addition that the language exposes the needed stack size.
Re:Async-await on stable Rust by littlestymaar on Friday November 08, 2019 @12:02PM (#21484174)
It's a stackless coroutine. AFAIK, the term “green thread” is usually reserved to stackful coroutines, but I guess it could also be used to talk about any kind of coroutines.
Re:Async-await on stable Rust by rcxdude on Sunday November 10, 2019 @02:07PM (#21499548)
It's more efficient (potentially substantially more so). In a typical threaded system you have some leaf functions which take up a substantial amount of stack space but don't block so you don't need to save their state between context switches. In most green threaded applications you still need to allocate this space (times the number of threads). The main advantage of this kind of green threads is you can seperate out the allocation which you need to keep between context switches (which is stored per task)versus the memory you only need while actually executing (which is shared between all tasks). For certain workloads this can be a substantial saving. In principle you can do this in C or C++ by stack switching at the appropriate points but it's a pain to implement, hard to use correctly (the language doesn't help you at all), and I've not seen any actual implementations of this.
Re:Async-await on stable Rust by dunkelheit on Friday November 08, 2019 @07:54AM (#21481765)
Kinda like a stack frame, but more compact and allocated in one go beforehand.
Re:Async-await on stable Rust by gpderetta on Friday November 08, 2019 @06:56AM (#21481468)
because of syntactical restrictions of how await work, at most you need to allocate a single function frame, never a full stack, and often it doesn't even need to be allocated separately and can live in the stack of the underlying OS thread.
Re:Async-await on stable Rust by jstimpfle on Friday November 08, 2019 @07:19AM (#21481576)
So that async function cannot call anything else?
Re:Async-await on stable Rust by gpderetta on Friday November 08, 2019 @09:42AM (#21482560)
They can, but the function itself cannot be suspended by calling something else (i.e. await being a keyword enforces this), so any function that is called can use the original OS thread stack. Any called function can in turn be an async function, and will return a future[1] that in turn capture that function stack. So yes, a chain of suspended async functions sort of looks like a stack, but its size is known a compile time [2].

[1] I'm not familiar with rust semantics here, just making educated guesses.

[2] Not sure how rust deals with recursion in this case. I assume you get a compilation error because it will fail to deduce the return value of the function, and you'll have to explicitly box the future: the "stack" would then look a linked list of activation records.

Re:Async-await on stable Rust by littlestymaar on Friday November 08, 2019 @09:47AM (#21482613)
Async function don't really call anything by themselves: the executor does, and all function called in the context of an async function is called on the executor's stack. You just end up with a fixed number of executors running on their own OS thread with a normal stack.
[dead]
Re:Async-await on stable Rust by jstimpfle on Thursday November 07, 2019 @03:02PM (#21476778)
What gives you doubts I'm serious?
Re:Async-await on stable Rust by weberc2 on Thursday November 07, 2019 @03:49PM (#21477111)
Proposing asm and deprecated APIs, "just use a portable wrapper library" as though there are any such (good) libraries, type safety, etc.
Re:Async-await on stable Rust by jstimpfle on Thursday November 07, 2019 @04:40PM (#21477598)
The fact is, all "green threads" do is swapping between call stacks and call register sets. The call stack is usually just a register itself. How hard can it be to save a register and restore it later?

> as though there are any such (good) libraries

Win32 Fibers are extremely easy to use (and not deprecated). I used them once, it was a blast. I wrote a simple wrapper around them in less than 50 lines of straightforward code, exposing maybe a struct and 3-4 function calls. Never had any problems.

That makes writing your own little non-preemptive scheduler extremely easy as well. That might be as little as another 50-100 lines of code. So you get a lot of control at almost no price.

POSIX C has swapcontext() / setcontext() which must be pretty similar (although I'm not sure I've ever used it) and if I understand correctly it was only obsoleted due to a syntactical incompatibility with C99 or something like that...

The only problem here is that green threads are a little bit of an awkward computational model in many cases. But that's not a syntax problem.

Re:Async-await on stable Rust by Rusky on Friday November 08, 2019 @12:12PM (#21484286)
An implementation of green threads-style register saving cannot be as memory-efficient as async/await, because it's more dynamic.

Green threads must save all (callee-save) registers, support arbitrary dynamic stack growth, and work with unsuspecting frames in the middle of the stack.

Async/await has the "thread" itself do all the state saving, so it can save only what's actually live. It computes a fixed stack size up front, so can be done with zero allocation. And because all intermediate stack frames are from async functions, they can also be smaller- transient state can be moved to the actual thread stack, potentially shrinking the persistent task state.

(This also has benefits for cancellation- there is no need to use the exception/stack unwinding ABI to tear down a "stack," because it is instead a normal struct with normal drop glue.)

Re:Async-await on stable Rust by jabl on Friday November 08, 2019 @11:23AM (#21483694)
> Win32 Fibers are extremely easy to use (and not deprecated).

Apparently MS is considering depreciating them: http://www.open-std.org/JTC1/SC22/WG21/docs/papers/2018/p136...

That's in general an interesting paper about M:N threads, fibers, green threads, or whatever you want to call them.

Re:Async-await on stable Rust by jstimpfle on Friday November 08, 2019 @11:38AM (#21483878)
Yeah, so what they write in section 3.1 is basically what I stated elsewhere on this comments page. It's probably not that Win32 Fibers is a bad implementation of Green Threads, but more that Green Threads is an awkward model to work with in many cases.

> Current recommendation is to avoid using fibers and UMS. This advice from 2005 remains unchanged: “... [I]nstead of spending your time rewriting your app to use fibers (and it IS a rewrite), instead it's better to rearchitect your app to use a "minimal context" model - instead of maintaining the state of your server on the stack, maintain it in a small data structure, and have that structure drive a small one-thread-per-cpu state machine.”