I think Project Loom would fit. It's a remarkably sane approach:

https://cr.openjdk.java.net/~rpressler/loom/Loom-Proposal.ht...

As a Java developer I'm really looking forward for Project Loom. I think it's a great approach that avoids the pitfalls of the two-colored functions approach.

However, Project Loom doesn't fit into the "zero cost abstraction" paradigm of rust. Project Loom requires quite a lot of support form the JVM runtime:

> The main technical mission in implementing continuations — and indeed, of this entire project — is adding to HotSpot the ability to capture, store and resume callstacks not as part of kernel threads. JNI stack frames will likely not be supported.

It also still require manual suspension so JIT compiled tight loops will likely still cause a problem.

I think that that makes it fairly similar to Rust's approach then... compiler support for suspension, and cooperative scheduling.

Clojure's pmap function (parallel map) is pretty cool: http://clojure.github.io/clojure/clojure.core-api.html#cloju...

There is rayon in rust land. This is a pretty different (simpler) thing.

Rayon uses iterators. Iterators are the dual of first order functions. Iterators are superior because they are more general. (You can implement pmap using iterators. You cannot implement iterators using pmap.)

Does Rayon use iterators, or just iterator-like combinators? I don’t think a for-loop would work with Rayon, would it?

You can also definitely implement iterators using first-order functions:

    const iterate = array => {
      const step = index => ({
        value: array[index],
        next: () => step(index + 1)
      });
      return step(0);
    }

    // add some combinators on top

First order functions can in fact implement anything if you’re willing to accept some bad syntax and speed — that’s the Church-Turing equivalence.

Hm. You're right. I hadn't used Rayon in so long I forgot it is indeed iterator-like combinators using an entry point called `par_iter`.

Python's greenthread implementations are quite good, monkey-patching all calls to work asynchronously with low mental overhead and automatic library comparability (except for libraries that call native code obviously).

Of course Python generally fails to offer "the same kind of performance" for anything limited by CPU or memory, so it technically doesn't fit the description either.

> since hot loops can block a whole OS thread

Asking as a beginner, what does the above mean?

Not sure what does hot loop means, and why does it block Os thread

Go creates the illusion of preemptive multithreading by having implicit safe-points for cooperative multithreading. Each IO operation is such a safe-point. If you write an infinite loop like `for {}` where there are no IO operations in loop body, it will block indefinitely. This will prevent the underlying OS thread from being available to other goroutines. The same thing can happen even if you do have IO operations in there, but the work being performed is dominated by CPU time instead of IO.

Note that this is being fixed in the next release: https://golang.org/issue/10958

That's cool it's finally coming, this issue is open since 2015! I wonder which performance impact this will have.

Maybe this is a dumb question, but what if a language with implicit safepoints lets you opt out with something like a nosafepoint block/pragma?

An “OS thread” is the most basic part of the program that can actually do things. When it is blocked, it can’t do anything.

OS threads can be expensive, so libraries try to only create a few OS threads.

A “hot loop” (usually, I think, called a “tight loop”) is a loop that does a lot of work for a relatively large amount of time all at once. Things like “looping through a list of every building an Manhattan and getting the average price over two decades.”

With some code like networking, you end up having to wait on other parts of the computer besides the CPU often, for things like uploads and downloads.

“Asynchronous programming” tried to make it easy to keep the CPU busy doing helpful things, even while some of it’s jobs are stuck waiting on uploads/downloads/whatever. This keeps the program efficient, because it can do a little bit of many tasks at once instead of having to complete each task entirely before moving on to the next.

The problem comes when you have a tight loop in a thread that is mostly expecting to be doing asynchronous work around I/O or networking. It is basically trying to juggle with one hand. The program can’t multitask as well, and you end up having to wait longer before it can start each piece of work you want.

Hot loops are often "expensive", or where your program is spending a lot of its time.

If you're doing a hot loop, which may be synchronous, you will block the event loop attached to that OS thread, because a hot loop is presumably not yielding control until it is done.

hmm, why is that a bad thing, if your entire thread is synchronous shouldn't it technically be blocking? Or is OS thread the main thread?

The problem is that the async model is a form of cooperative multithreading, so if one computation runs for a long time without returning to the main event loop, it can increase the latency for responses to other events. E.g., if one HTTP request takes a long time to process, and many of the worker-pool OS threads are handling such a request, response time goes up for all the other requests. OS-level concurrency is preemptive (timesliced), so one busy thread doesn't block other requests, but of course with much higher overhead in other ways. Best practice is usually to keep event handlers on event-loop threads short and push heavy computations to other OS-level worker threads.

ah, that make sense, I think another user point out that spawning n goroutine does not actually spawn n physical threads but rather queue n task to m thread in the pool, so if we exhaust m thread, n-m task will be blocked.

Thanks for the explanation

I wonder what is the point where each trade off make sense (ie. what is consider heavy computation vs light computation, it probably is related to OS thread allocation time)

> interactions with native libraries are really slow because of that

Can you explain a bit? What is the connection between concurrency implementation (which I am assuming you are talking about multiplexing multiple coroutines over the same OS thread) and say slowness in cgo? Having to save the stack? I don't get it.

FFI is slow because the main Go compiler uses a different calling convention than everything else. I couldn't tell you if or how that's related to its concurrency features.

Its an important optimization otherwise each go routine would use a lot of memory, but its not required. The stack allocation strategy has changed a couple times in main compiler, gccgo originally support it and CGo functions behave like normal go mod the stack.

> Can you give one that reaches this goal?

First-class continuations. They are an efficient building block for everything from calling asynchronous I/O routines without turning your code into callback spaghetti, to implementing coroutines and green threads. Goroutines are a special and poorly implemented case of continuations. Gambit-C has had preemptive green threads with priorities and mailboxes at less than 1KiB per thread memory overhead for over a decade now, all built on top of first-class continuations.

http://www.iro.umontreal.ca/~gambit/Gambit-inside-out.pdf

First-class continuations are too unrestricted to match the performance of Rust-style async/await.

If you take on a few limitations you can get there, but then those are exactly the things people complain about.

It is worse than that. First-class continuations add a small overhead for stack operations, and they always have more memory overhead than callbacks/async transformation into callbacks. But the comment I was replying was about Go, not Rust. For the small overhead that first-class continuations have, they provide a simple and efficient way to handle coroutines, generators, asynchronous I/O, and green threads.

Erlang probably comes closest?

Not an Erlang user, but my understanding is that the Erlang VM (BEAM) schedules on function calls. Which works fine for that use, since Erlang does looping with tail calls, but is not a solution for procedural languages.

Erlang vm is fully preemptive and schedules on something called a reduction. You can call an external function with EFI that can screw things up, but otherwise it's not necessarily on what you might consider a function call .

In practice, it's enough to preempt at potentially-recursive function entry and loop backedges. A thread which doesn't recurse or loop has to come to an end pretty quickly, at which point the scheduler will get a go.

There's no fundamental difference between inserting a yield before a tail call and inserting a yield before a continue statement or closing bracket of the for block, is there?

Fundamentally no, but it's a more difficult thing to do in practice at least. Unless you want to give up other optimizations like loop unrolling and other things like that since you end up losing the fact that the loop exists after some optimization passes.

BEAM (Erlang VM) provides pre-emptive scheduling

True for functions written in Erlang/Elixir, but not in NIF functions implemented in C.

Thanks to dirty schedulers that is not a big issue: https://medium.com/@jlouis666/erlang-dirty-scheduler-overhea...

Thanks for the link. It's a very deep and interesting technical post.