I kinda agree but yielding only on I/O is not enough IMO. Sooner or later we'll want to do CPU intensive work in fibers as well, which in the current implementation will block all other fibers on the same thread. Like I mentioned in the article, I'd like to see work stealing between different threads as that would allow fibers to migrate away from threads stuck in CPU intensive work. An alternative way could be to adopt a model similar to Haskells lightweight threads, where the runtime forces a `yield` after N milliseconds (configurable). That would make sure that CPU intensive work would not block other fibers "too" much.
Then they added a guarantee that if your loop called a function, the scheduler would be able to make the goroutine yield. https://golang.org/doc/go1.2#preemption
This mostly worked although you could still have a CPU-bound thread not making any function calls. I also personally ran into a pathological issue where the scheduler was being invoked, but a heuristic kept the current goroutine running so others were still starved.
Finally they added true pre-emption (not yielding) in 1.14 https://golang.org/doc/go1.14#runtime it looks like it just sends signals and saves state.
Once nice thing is that if I understand the go runtime correctly, work stealing by scheduling goroutines on different threads has been a thing for a long time.
This limitation is present in every modern asynchronous programming system I've ever seen. They simply can't deal with long function calls. There's a loop under the hood which constantly executes queued functions. Any long-running function will stall the execution pipeline and increase latency. It's just like cooperative multitasking...
And for just semantically long running functions, it is not difficult to write a scheduler that tries to be "fair", however you want to interpret that. But of course it requires to fully embrace asynchronous programming, something that will probably difficult in Ruby.
Not really, they've just made it super implicit. Any FFI calls are now implicitly coloured, same with anything CPU-heavy.
Like their approach to type annotations, I think this will be a mistake in hindsight.
That said, you're right in that one may have to make some changes to code to get around some new problems. The problem with FFI/CPU-heavy functions is that they prevent _other_ fibers from being scheduled while they run.
If it becomes possible to implement work-stealing, then that would mitigate the problem. It would also be solvable by sprinkling "yield to scheduler" calls throughout such functions. Annoying and not always possible, but, since in these cases the signature would not change, technically not coloring.
[1]: https://glyph.twistedmatrix.com/2014/02/unyielding.html
Golang does essentially the same thing and uses channels instead of "futures". However, some APIs will have functions that return a channel that you receive a message on later.. Which is a lot like a future. And if you provide a more simple non-channel API along side(instead of making the consumer wrap your calls in a Go routine or always use a channel), you now have "colored" functions.
I believe this is just a solution to a scheduling problem(pre-emptive vs cooperative).
This is an aside and possibly pedantic, but everyone seems to have forgotten that the whole idea of futures was originally that they weren't some different type. They were the same type, but blocked transparently on demand and had no `get` operation.
The construct that came before futures, called eventual-values, had the different-type thing. The only difference of futures, the big idea, was they said 'maybe we can get rid of that part'.
> These “futures” greatly resemble the “eventual values” of Hibbard’s Algol 68 [39]. The principal difference is that eventual values are declared as a separate data type, distinct from the type of the value they take on, whereas futures cannot be distinguished from the type of the value they take on.
(Halstead, 1985)
But now people write types like `Future[T]`... which completely misses the point! If you start to write `Future[...` then think to yourself... this isn't a future. And that's how you get this colouring problem you describe... it was already fixed and people have forgotten all about it.
Yes - because it means existing code that expects a T cannot accept a Future<T>, which is the colouring problem being discussed.
You end up with half your code accepting futures and the other legacy half not accepting them and there's a divide. If you want to use the legacy code you have to force the future potentially before you really needed it.
> I don’t see the alternative.
The alternative is wherever a function accepts a concrete value T t, also let it accept a future value T t. When t is actually used, then block as if get was called.
First of all, this is a programming language specific problem, not a general one. A good language allows you to write:
someFuture.map(someFunction)
Yes, you cannot write someFunction(someFuture), but that is a good thing, since asynchronous execution can very well change the behaviour of the program. Hence, I want the compiler (if there is one) to warn me if a variable that I pass to a function is now a Future instead of a plain value. I then map over the Future and I'm good. Or, maybe I want to wait for completion of the future and I'm good.
In both cases, there is no reason to write a second version of "someFunction".
Therefore...
> If you want to use the legacy code you have to force the future potentially before you really needed it.
... this is only true in incapable programming languages.
You need support in the language to do this. If you're writing a library routine from scratch as Future<... with the compiler knowing nothing about how to handle it specially then you aren't getting a future as they were originally envisioned and you're missing out.
I don't think so - let me try to make it more clear, hold on.
> You need support in the language to do this. If you're writing a library routine from scratch as Future<... with the compiler knowing nothing about how to handle it specially then you aren't getting a future as they were originally envisioned and you're missing out.
You seem to assume that the programming language needs to know of the existence of Futures to help. But that is not the case and probably the reason for the misunderstanding.
It is sufficient that the language supports generics/type-parameters as well as a lightweight syntax (such as in the example I gave) and optimally higher kinded types or a similar mechanism.
If that is given, the language can be totally agnostic of Futures or async programming in general and it is easily possible to define a Future type as a library-type and use it in the way I described. Scala is a good example of such a language.
someFuture.map(someFunction)
This blocks the entire of someFunction until all the values are ready. You haven't adapted someFunction to work with futures at all - you're simply not running it until all the futures are gone and only normal values remain which you then pass into it!
Imagine someFunction comes from a library so cannot be modified trivially, takes three values, and can run successfully with the first two completed values while the third completes. What would you write with map to do that?
So then...
> Imagine someFunction comes from a library takes three values, and can run successfully with the first two completed values while the third completes. What would you write in someFuture.map(...) to do that?
That is a good example actually. I believe that the user (at the callsite) should both _know_ if the library operates in an async way and/or runs code in parallel _and_ the user should be in control of choosing his desired behaviour.
Hence, with a library function that just expects 3 values, this is not possible and I would say the design isn't great.
Here is how I would write it in scala-ish pseudocode:
// library - two separate functions instead of one
def calcX(a, b): X
def calcY(x, c): Y
// One "combined" synchronous function can be created easily if desired:
combinedCalc = calcX.andThen(calcY)
// all given values are (lazy) futures
aFuture = future(...)
aFuture = future(...)
aFuture = future(...)
// callsite - async but _no parallel execution_
xFuture = (aFuture, bFuture).mapN(calcX) // This is actually valid Scala syntax. Returns a future
yFuture = (xFuture, cFuture).mapN(calcY)
// alternative Scala syntax
yFuture = for {
a <- aFuture
b <- bFuture
x <- calcX(a, b)
c <- cFuture
} yield calcY(x, c)
yFuture = for {
a <- aFuture
c <- cFuture
b <- bFuture
x <- calcX(a, b)
} yield calcY(x, c)
How to run it in parallel?
xFuture = (aFuture, bFuture).parMapN(calcX)
yFuture = (xFuture, cFuture).parMapN(calcY)
Sorry the wall of code, but it shows that the code can look rather short even without future or async specific syntax support from the language.
The important part here is that it allows me to have fine-grained control over the execution of my code. There are a lot of situations where I probably don't care and it might be nice to have a way to say "Hey compiler, I don't care, just run that code in the way you think it's best", similar to what an SQL planner does with my queries.
But often I want control. For instance, in your example: what happens if the library decides to always evaluate "c" eagerly even though "a" and "b" often fail and the whole calculation is aborted? That would be a waste of CPU. But worse: the library might change its behaviour any time and keep the same function signature.
While that might be not so bad for mere calculations, what if we start to think about actions? Such as database or file or network IO - or even in-memory transactions that should be atomic. How about _streams_ of these actions, such as running an action every second and then combining this with other streams of actions? Losing control over the fain grained execution will result in terrible hard to debug and fix bugs due to concurrency.
This is just not a true statement - and that was the point of the original thread - the original paper on futures had them implemented transparently and able to be passed down into library code not expecting futures. It's also how Haskell's par works today. So it demonstrably can be implemented.
> I believe that the user (at the callsite) should both _know_
Ok but this is just a 'don't do that' argument. If a user tells us they have a good reason to do it, I'm interested in how we can enable them to do that.
> with a library function that just expects 3 values, this is not possible
But it is in Haskell using par, and other languages with more traditional futures.
> Here is how I would write it
Great... but you've rewritten the library code with 'two separate functions instead of one.' My original point was 'because it means existing code that expects a T cannot accept a Future<T>' and you're proving that back to me so hard that you've accepted you need to completely rewrite the code to do it!
Languages with traditional futures would let you just call the original, unmodified library code, with future values instead of actual values, and it would just work.
If you had library code like this, you could pass in future values of T for a, b, and c, and it would execute the expensive_operation on a and b as soon as it could, while c completes.
def foo(T a, T b, T c)
expensive_operation(a, b) + expensive_operation(b, c)
end
Yes this is a problem with transparent futures, as is the ordering issue you raised. But I wasn't arguing they didn't have their own issues. You're saying it's impossible and that is demonstrably false, because languages do do it.
(I did half of my PhD on these topics, and the other half on Ruby.)
Okay, technically you are right - however it changes the semantics of the program. I admit that I didn't mention this constraint though.
> Great... but you've rewritten the library code with 'two separate functions instead of one.' My original point was 'because it means existing code that expects a T cannot accept a Future<T>' and you're proving that back to me so hard that you've accepted you need to completely rewrite the code to do it!
> Yes this is a problem with transparent futures, as is the ordering issue you raised. But I wasn't arguing they didn't have their own issues.
I don't disagree with any of that. However, what you originally said:
> > Are you saying that generics on futures are bad?
> Yes - because it means existing code that expects a T cannot accept a Future<T>, which is the colouring problem being discussed.
And you imply that transparent futures are better than the ones with generics. So where we disagree is on what is worse. I am convinced that the problems (and there are certainly a lot of unknowns) you introduce with transparent futures is much worse than the performance you might gain in certain cases and the easier syntax in many cases.
I believe it's important to expose the fact that an asynchronous(potentially parallel) task is doing something for you. The runtime semantics of an assignment would appear the same in code by be very different behind the scenes otherwise. Also, a lot of code is written to manage those tasks in use-case specific ways. How would you race or cancel transparent futures?
Of course because Rust's runtime or lack thereof aims to be zero overhead a Future could never be without the poll function.. right? You need a runtime to make regular functions accept these magic opaque future values.
Yes it requires some kind of special compiler or runtime support.
Looking at the API for Fiber.. It looks like "resume" returns the yield(generator) or fiber return value? So Fibers can be treated as a sort of Promise? As soon as you start returning fibers from methods you have the coloring problem..
This is huge. They solved the function-coloring problem by deciding that all functions are now potentially async. It makes it more likely that the ecosystem as a whole actually becomes async-compatible. I wish Python had taken this approach, though I understand why they didn't.