News for hackers. Stuff that matters.
Re:C++ coroutines: Initial implementation pushed to master by tus88 on Sunday January 19, 2020 @02:21AM (#22089957)
There have been library implementations of this for years. And they are a far cry from true lwt aka golang.
Re:C++ coroutines: Initial implementation pushed to master by pjmlp on Sunday January 19, 2020 @04:18AM (#22090348)
Go routines are a tiny subset of what coroutines allow in other languages, because Go doesn't have the concept of generators/continuations nor exposes the low level runtime management APIs.
Re:C++ coroutines: Initial implementation pushed to master by gpderetta on Sunday January 19, 2020 @05:02AM (#22090511)
Technically goroutines are equivalent to one shot continuations, while generators, lacking the ability to capture a call stack are not (they capture the return continuation, but not any arbitrary one).

So no, go coroutines (or any stack full coroutines) are actually more powerful than thise exposed in other languages.

Re:C++ coroutines: Initial implementation pushed to master by anonymoushn on Sunday January 19, 2020 @02:23AM (#22089963)
Goroutines are not copyable so they're not really a substitute for byuu/co
Re:C++ coroutines: Initial implementation pushed to master by tomcam on Sunday January 19, 2020 @02:26AM (#22089974)
I think my mind is about to be blown. What does copyable mean when it comes to coroutines?
Re:C++ coroutines: Initial implementation pushed to master by byuu on Sunday January 19, 2020 @04:37AM (#22090425)
Serialization [1].

Say you want to take a snapshot of where your program is, run the code some more, and then later on restore execution back to that snapshot.

In the parlance of video games and emulators, this would mean save states. You could use these save states to implement speculative execution to eg remove input latency from games using a method called run-ahead [2].

With state machines, this is trivial: just record all the state machine variables. With coroutines / cooperative threading, it's not: the state is all stored on the stack frame.

My stackful coroutine library can do this by essentially memcpy'ing the stack frames themselves, but it comes with some seriously negative caveats: the biggest is you can't save this to disk, because on the next program run, ASLR, etc would invalidate the pointers in the stack frame. In fact, you can't even safely release and allocate new memory between a serialize and unserialize point, or the stack frames may have dangling pointers in them. In practice, this is acceptable and workable for features like real-time rewind and run-ahead.

Library support for C++ coroutine serialization would be trivial since they are stackless. Getting around the limitations I mentioned for stackful coroutines would be immensely more challenging.

[1] https://byuu.net/design/cooperative-serialization

[2] https://byuu.net/input/run-ahead

Re:C++ coroutines: Initial implementation pushed to master by speedplane on Sunday January 19, 2020 @04:42AM (#22090449)
> Getting around the limitations I mentioned for stackful coroutines would be immensely more challenging.

Challenging but it's definitely been done before. The system below [1] (disclosure: I worked on it years ago), stores every memory write in a separate buffer on the assembly level. You need to modify the compiler for this, and it cuts performance roughly in half, but it allows you to step backwards in code and perform sophisticated performance analysis.

[1] https://www.ghs.com/products/timemachine.html

Re:C++ coroutines: Initial implementation pushed to master by byuu on Sunday January 19, 2020 @04:56AM (#22090488)
Oh certainly, it's just beyond what I can do as a C library. I've also seen another approach that implemented a customized libc replacement to allow this to work. But that and modifying the compiler itself are really extreme steps.

It's still doable, and it's definitely worth doing, but I understand the uses I've mentioned are rather niche, so it's unlikely to occur. For one, we still don't have built-in stackful coroutine support even in C++20.

Re:C++ coroutines: Initial implementation pushed to master by speedplane on Sunday January 19, 2020 @05:04AM (#22090517)
> modifying the compiler itself are really extreme steps

Yes, it's not for the average developer. Yet it should be incumbent on the tools builders. I'm actually surprised that most compilers/interpreters don't have a "step backwards" flag that you can turn on during compile/run-time. It would be a helpful feature to expert and novice developers alike, but hasn't really caught on outside of niche areas.

Re:C++ coroutines: Initial implementation pushed to master by aidenn0 on Tuesday January 28, 2020 @12:50AM (#22166783)
Can't you just disable ASLR?
Re:C++ coroutines: Initial implementation pushed to master by anonymoushn on Sunday January 19, 2020 @02:54AM (#22090067)
A description of one thing you can do with this is at https://github.com/moonjit/moonjit/issues/84

It's not a common feature. Usually it's present in a language that has "continuations" and the api is something dreadful like call/cc. But you can use it in Lua with the above mentioned patch, and you can use it in byuu/co with memcpy.

Re:C++ coroutines: Initial implementation pushed to master by simias on Sunday January 19, 2020 @07:24AM (#22091041)
I object to the use of "dreadful" for call/cc, it's just a different paradigm that requires a different way of thinking but it's not a bad thing per-se. It's not like goto or setjmp which are fundamentally unsound constructs that completely break the flow of the program.
Re:C++ coroutines: Initial implementation pushed to master by anonymoushn on Sunday January 19, 2020 @09:38AM (#22091630)
When designing these primitives, I think there are a few considerations: When creating a new execution context, which one should begin executing, and which one should be bound to a local variable in the executing one? Should there be any primitive that switches to an already-existing execution context without creating a copy of it? How many primitives should there be?

call/cc answers: "The new execution context should begin executing and the old one should be bound to a local variable, no, one"

As a person who occasionally explains these things to people who have never been exposed to them, I strongly prefer: "The old execution context should continue executing and the new one should be bound to a local variable, yes, as many as we need to produce the preceding answers"

I think without trying too hard you can get the number of primitives in the latter case down to three, "create", "copy", and "switch." If you wanted to bug your users you could get it down to two.

Re:C++ coroutines: Initial implementation pushed to master by wbl on Sunday January 19, 2020 @09:50AM (#22091684)
Call/cc is not very composeable.
Re:C++ coroutines: Initial implementation pushed to master by tsimionescu on Sunday January 19, 2020 @05:19PM (#22094530)
Well, call/cc usually makes it impossible to add a construct like 'finally' to a language, so it does at least block some options.
Re:C++ coroutines: Initial implementation pushed to master by anonymoushn on Sunday January 19, 2020 @10:38PM (#22096129)
Depending on what you want from this, you can maybe use dynamic-wind. But yeah.
Re:C++ coroutines: Initial implementation pushed to master by gumby on Sunday January 19, 2020 @08:15AM (#22091271)
In c++20 lambdas are now copyable and there’s no reason the routines calling co_awai, co_yield, and co_return can’t be lambdas perhaps called by lookup.
Re:C++ coroutines: Initial implementation pushed to master by tus88 on Sunday January 19, 2020 @03:07AM (#22090113)
Lol continuations will blow your mind. "Returning" to somewhere else than you came? Humbug!
Re:C++ coroutines: Initial implementation pushed to master by rakoo on Sunday January 19, 2020 @03:14AM (#22090137)
Unless I'm mistaken you can already do that with go's channels
Re:C++ coroutines: Initial implementation pushed to master by tus88 on Sunday January 19, 2020 @03:18AM (#22090152)
Hint: I don't mean returning data ;-)
Re:C++ coroutines: Initial implementation pushed to master by rakoo on Sunday January 19, 2020 @04:48AM (#22090461)
And you don't need to return data with channels, even though you typically do. They are primarily a tool for controlling flow with concurrent goroutines.

Do you have an example of what you have in mind so we can see more clearly what it is about?

Re:C++ coroutines: Initial implementation pushed to master by anonymoushn on Sunday January 19, 2020 @04:54AM (#22090480)
byuu's comment in this comment tree gives a good example.
Re:C++ coroutines: Initial implementation pushed to master by AnanasAttack on Sunday January 19, 2020 @03:04PM (#22093634)
C++ coroutines are stackless
Re:C++ coroutines: Initial implementation pushed to master by halayli on Sunday January 19, 2020 @02:35AM (#22090007)
Is that so? Please elaborate
Re:C++ coroutines: Initial implementation pushed to master by civicsquid on Sunday January 19, 2020 @03:31AM (#22090197)
It looks like Facebook's `folly` library had an implementation of coroutines available in their experimental directory since mid-2018. There was still some evolution going on after it was pushed though (and it seems like there still is today).

Not sure if that helps the argument that this has been around for "years" (since it's still in the experimental directory), but I guess there were some libraries trying to accomplish this.

https://github.com/facebook/folly/tree/master/folly/experime...

Re:C++ coroutines: Initial implementation pushed to master by pjmlp on Sunday January 19, 2020 @04:20AM (#22090355)
The only implementation that I would consider around of years, were the C++/CX extensions, and even those aren't 100% like ISO C++ ones ended up looking like, although Microsoft was a big contributor to its design.
Re:C++ coroutines: Initial implementation pushed to master by tus88 on Sunday January 19, 2020 @02:49AM (#22090048)
https://en.m.wikipedia.org/wiki/GNU_Portable_Threads

N:M threading from 1999!

And there are many more recent examples too.

Re:C++ coroutines: Initial implementation pushed to master by armitron on Sunday January 19, 2020 @10:17AM (#22091832)
It's not N:M with arbitrary M, but N:1 (mentioned in the link you posted).

There is no parallelism whatsoever in GNU Pth.

You only get a single kernel thread and "green" userland threads cooperatively scheduled on top of that single kernel thread.

That's it.