Re:The Rust standard library no longer has any scheduling ba...
by qznc on Sunday December 29, 2013 @05:40AM
(#6979136)
C integration is the killer for N:M threading. C libraries do blocking IO, are not wrapped into a VM, are not compiled for segmented stacks, use thread-local storage, etc. You can work around this problems, which increases complexity. For close C integration, you are basically forced to adopt Cs view of threading.
Re:The Rust standard library no longer has any scheduling ba...
by jlouis on Sunday December 29, 2013 @01:01PM
(#6980467)
This is why the R17 release of Erlang might get "dirty schedulers" for this kind of work. Essentially it gives you background thread pools where such work can be carried out.
Currently, you can call C functions directly, but it is not meant for long-running tasks which can block and so on.
Re:The Rust standard library no longer has any scheduling ba...
by wcummings on Sunday December 29, 2013 @06:48PM
(#6982162)
If E17 includes this AND maps I might cry (':
Re:The Rust standard library no longer has any scheduling ba...
by wcummings on Sunday December 29, 2013 @06:44PM
(#6982136)
You can create threads with enif_thread_create and build a queue to handle long running tasks (or a pool of threads), and use enif_send to return the result. Not quite as hacky as it sounds, since afaik port drivers rely on message passing to return values to the VM as well. Maybe not ideal, but better than naively bumping reductions. I haven't played with enif_timeslice yet, so I'm not sure how it compares (it's newer than NIF threads).
* IIRC each new process consumes 284 bytes so it's not "spending lots of memory on creating stacks for your green threads"
* Synchronization is a non-issue as everything is message-passing.
* Disk I/O is blocking, but the VM has dedicated disk threads.
* Dealing with 3rd party code is a problem, which is why relatively few libraries exist for Erlang. However, there is some hope now that NIFs (native interface functions, i.e. C code) can interact with the scheduler, i.e. report how much time has been used and return control if necessary.
You are right that "Building a correct highly concurrent scheduler is no easy task." The Erlang scheduler was not an easy task, and has received a ton of work from extremely talented engineers over the course of decades. Definitely worth a look to see how N:M can be made to work well.