News for hackers. Stuff that matters.
Re:C# functional language extensions and Erlang-like concurr... by im_down_w_otp on Sunday January 29, 2017 @01:12PM (#13515631)
It's not clear to me from a cursory look through the docs how the scheduling system works. "Erlang-like" concurrency works so well predominantly because the runtime uses preemptive scheduling instead of cooperative (which is what basically everything else uses, unfortunately).
Re:C# functional language extensions and Erlang-like concurr... by Matthias247 on Sunday January 29, 2017 @05:06PM (#13517015)
I think it works similar to how Akka actors work. The actors are purely reactive and their message processing functions are called whenever a new message arrived. You shouldn't block them (e.g. to wait for a response from another actor), because that blocks the actor from doing other things as well as an OS thread. Instead you should return and wait for being called again with a response message. This makes some things a little bit harder to write, since you will need some kind of state machine and store the current state of actors in a more explicit fashion. However you can also achieve a higher reactivity with this kind approach, so it's a trade off. In C# you could basically get some kind of best-of-both-worlds by allowing async handlers (that return Task<T>), which means no thread would be blocked if the message handler would do a long running async operation. I think that was discussed for Akka.Net, but I don't know if it's now supported or not.
Re:C# functional language extensions and Erlang-like concurr... by jahewson on Sunday January 29, 2017 @01:42PM (#13515797)
That's incorrect, most scheduling systems, including OS threads are preemptive. Erlang's scheduler itself is actually cooperative - though it can still preempt the "processes" it schedules.

Erlang's message passing system is probably more important to the success of its concurrency model than the specifics of the scheduler. As proven perhaps by the success of Akka and other actor-based systems built on a different threading model.

Re:C# functional language extensions and Erlang-like concurr... by im_down_w_otp on Sunday January 29, 2017 @04:02PM (#13516625)
I was talking about scheduling in actor-pattern frameworks and non-OS runtime environments (JVM, C++, Python, etc. etc. etc.).

Erlang's scheduler is preemptive. Erlang processes can be preempted by the scheduler in the middle of execution and moved to the back of the scheduler queue or to another scheduler thread. There is no requirement for a process itself to yield time back to the scheduler to make sure that other processes receive some execution time. This is one component of how it achieves "soft-realtime" semantics. Eight processes, on a system with 8 scheduler threads, running in a tight loop cannot cause all other processes in the Erlang VM to be blocked. Other processes will be given time.

The only thing that violates this guarantee is long-running or blocking code executing on the native side of the NIF boundary.

Akka, goroutines, Cloud Haskell, etc. are exactly the kinds of things I'm talking about when I say I don't understand the point of actor frameworks running in environments that can't preempt actor execution. You had better hope all your actor implementations are well-behaved or that the implementations you're going to import from someone else are all well behaved... otherwise you're going to lockup your scheduler's thread pool.

Re:C# functional language extensions and Erlang-like concurr... by bitwalker on Sunday January 29, 2017 @01:51PM (#13515855)
I think what he meant is that from a programmer's point of view, Erlang appears to be preemptive (there is no explicit yielding of control, ala golang or lua, the two I can think of off the top of my head), even if the scheduler itself is cooperative. The place where this most obviously reveals itself is with NIFs, where the scheduler can no longer preempt the process, and is blocked until the NIF finishes executing. At least that's my understanding.
Re:C# functional language extensions and Erlang-like concurr... by jahewson on Sunday January 29, 2017 @01:55PM (#13515872)
Right, that's exactly what I said above. But OS threads are preemptive in exactly the same way. It is not unique to Erlang as claimed, in fact it's the norm.
Re:C# functional language extensions and Erlang-like concurr... by dxhdr on Monday January 30, 2017 @01:03AM (#13519033)
I think you're confusing the issue; OS processes are preemptive, threads within a process are not. Preemptive threads are not the norm on Unix / Linux / Windows systems. The BEAM VM implements its own threads which allows for them to be preemptively scheduled.
Re:C# functional language extensions and Erlang-like concurr... by dxhdr on Monday January 30, 2017 @01:09AM (#13519051)
> That's incorrect, most scheduling systems, including OS threads are preemptive. Erlang's scheduler itself is actually cooperative - though it can still preempt the "processes" it schedules.

OS processes are preemptive, however process threads are not. I'm not sure what you are trying to say about Erlang's scheduler itself being cooperative. It is a preemptive scheduler.

> Erlang's message passing system is probably more important to the success of its concurrency model than the specifics of the scheduler.

I disagree, preemptive concurrency is a fundamental piece of BEAM's success. Preemption gives Erlang its fault tolerance and low latency guarantees. If by message passing you mean immutability, then perhaps they are of equal importance.

Re:C# functional language extensions and Erlang-like concurr... by rdtsc on Sunday January 29, 2017 @07:49PM (#13517834)
> That's incorrect, most scheduling systems, including OS threads are preemptive. Erlang's scheduler itself is actually cooperative

From what I know Erlang's scheduler preempts processes it executes. Can you explain what you mean by it being "cooperative". It precisely useful because it is not cooperative. And processes don't have to "yield to" the scheduler.

Re:C# functional language extensions and Erlang-like concurr... by dozzie on Sunday January 29, 2017 @03:17PM (#13516333)
Not quite, as a sibling comment states. Everything today's operating systems provide in scheduling (processes and OS-level threads) is preemptive. Erlang's (BEAM's) scheduling is cooperative, but Erlang compiler inserts yield points in appropriate places, so it actually looks and feels like preemptive scheduling.
Re:C# functional language extensions and Erlang-like concurr... by im_down_w_otp on Sunday January 29, 2017 @04:23PM (#13516769)
Erlang processes can be interrupted by the scheduler. Saying that Erlang scheduling is "cooperative" because it uses reduction counts as one (among many) hints about how to decide what to schedule is like saying that preemptive OS kernel schedulers are "cooperative" because they use mechanisms like time-slices, etc. to attempt to balance execution resources.

The Erlang VM can intercede into a running process, and also does this with some of it's BIFs implemented in C.

Re:C# functional language extensions and Erlang-like concurr... by dozzie on Sunday January 29, 2017 @05:06PM (#13517019)
> Erlang processes can be interrupted by the scheduler.

From what I know, they cannot, not at arbitrary places, and that's why the scheduler is a cooperative one.

Re:C# functional language extensions and Erlang-like concurr... by im_down_w_otp on Sunday January 29, 2017 @11:26PM (#13518651)
> "Scheduling is pre-emptive. Regardless of priority, a process is pre-empted when it has consumed more than a certain number of reductions since the last time it was selected for execution."

From the Erlang docs on process priority.

The BEAM VM can kill, halt, or suspend a running process at anytime from outside the process. It does not need to wait around for the process to exceed its reduction count limit.

Some of this capability is even exposed to user-land applications. The exit/2 function builds on top of this capability when sending in th atom 'kill' as the second argument. As do the suspend_process/1, suspend_process/2, and resume_process/1 functions.

The reduction count system is just there as a kind of fair-scheduling determinism semantic. Exhausting reduction allocations is not the only way the VM has to interact with running processes.

Re:C# functional language extensions and Erlang-like concurr... by dozzie on Monday January 30, 2017 @05:50AM (#13520027)
>> "Scheduling is pre-emptive. Regardless of priority, a process is pre-empted when it has consumed more than a certain number of reductions since the last time it was selected for execution."

> From the Erlang docs on process priority.

Well, apparently you're right. I must take back what I said about Erlang scheduler.

Re:C# functional language extensions and Erlang-like concurr... by louthy on Monday January 30, 2017 @05:40PM (#13525772)
Author here. "Erlang-like" was mostly used for programmers that aren't aware of actor systems, or the current de facto standard language for actors. It's not a statement on its competitiveness with Erlang.

The system simply uses the TPL for scheduling (which is actually very good).