News for hackers. Stuff that matters.
Re:A deep dive into Multicore OCaml garbage collector by lobster_johnson on Sunday July 16, 2017 @09:14AM (#14781855)
In Go you can just do "go doWork()" and the scheduler runs the function as a coroutine on some kernel thread, thus allowing your program to make use of all cores.

The ability to just throw goroutines at the scheduler changes how you approach writing programs. For example, say you want to process an input that consists of lots of individual records. You can run the input stream in one goroutine, then spawn a bunch of worker goroutines that each receives records on a channel for processing. Since Go uses real threads and doesn't have a global lock, you can get nearly linear improvement in throughout here.

You wouldn't do this in, say, Ruby or Python, where the concurrency situation is about the same as in OCaml. Ruby has threads, like OCaml, but their slowness means they're not really usable in the same way as goroutines.

Parallelizing apps by forking child processes and communicating input and results via pipes is something I've done a lot in Ruby, and it's a really awkward, heavy-handed concurrency model.

Re:A deep dive into Multicore OCaml garbage collector by rbehrends on Sunday July 16, 2017 @10:30AM (#14782095)
Go doesn't really have a good shared memory story, either. Shared memory support exists, but it's not really any safer than in C or Java. If that were good enough, then Ocamlnet's netmulticore library would already fit the bill. But most people who want shared memory support want something that's both safer and higher level.

Go really is designed around message-passing as its primary means of coordination; the key part here is having nice abstractions, which is where Go's strengths lie and which revolve primarily around `select`. But writing higher-level abstractions over processes and FIFO channels isn't terribly hard, as long as you have serialization built in (which Python, Ruby, and OCaml all do).

Re:A deep dive into Multicore OCaml garbage collector by JacobiX on Sunday July 16, 2017 @09:41AM (#14781932)
In OCaml there is Lwt. It provides very light-weight cooperative threads. The context switches are very fast and composing cooperative threads allows the writing of highly asynchronous applications. For instance, in the versions of Go prior to 1.5 goroutines used to run on a single core. And as of 1.5 the variable GOMAXPROCS can be set to the number of cores. EDIT:clarify the difference between concurrency and parallelism.
Re:A deep dive into Multicore OCaml garbage collector by jzelinskie on Sunday July 16, 2017 @01:08PM (#14782950)
>For instance, in the versions of Go prior to 1.5 goroutines used to run on a single core. And as of 1.5 the variable GOMAXPROCS can be set to the number of cores.

This is not true. Prior to Go 1.5 GOMAXPROCS defaulted to 1, but could always be set to use any number of cores. In Go 1.5, the default value for GOMAXPROCS was changed to the number of "cores" available on the machine. Prior to then many applications were setting GOMAXPROCS based on CLI flags or just hard-coding the future behavior with `runtime.GOMAXPROCS(runtime.NumCPU())`.

Re:A deep dive into Multicore OCaml garbage collector by JacobiX on Sunday July 16, 2017 @02:21PM (#14783366)
My bad, you are correct. I just wanted to point out that one can have concurrency without parallelism.
Re:A deep dive into Multicore OCaml garbage collector by lobster_johnson on Sunday July 16, 2017 @09:58AM (#14781979)
Lwt looks nice, but also seems a lot more intrusive to me than, say, Go. It's based on promises (so it's very explicit), and if I remember correctly, in order to do any I/O you have to use the Lwt mechanisms instead of the standard ones (which block).

How does Lwt interact with third-party libs that aren't written to use Lwt? I know from Ruby that libevent-type stuff such as Eventmachine doesn't combine all that well with non-async code.

Go made a good call in making all I/O blocking. That means code is easy to read and write. It comes with downsides, such as that you have to use goroutines, but for most developers it's a more productive concurrency model. (Unfortunately there's no alternative; you can build a blocking system out of a non-blocking one with no overhead, but the opposite is not true. When I first started with Go I was surprised that you can't "select{}" on a file descriptor or socket, or indeed any custom implementation, in the same way that "range" is magical and only works on built-in types.)

Re:A deep dive into Multicore OCaml garbage collector by kcsrk on Sunday July 16, 2017 @10:25AM (#14782072)
With algebraic effect handlers (the concurrency story of multicore OCaml), you can write your I/O code in direct-style, but retain the advantage of non-blocking I/O. As such, you have the advantage of goroutines, but this is also an opt-in. The asynchronous I/O is implemented as a library and is not baked into the language, and hence you can write your own I/O library. For more details, see our recent draft paper: http://kcsrk.info/papers/system_effects_may_17.pdf.
Re:A deep dive into Multicore OCaml garbage collector by JacobiX on Sunday July 16, 2017 @10:11AM (#14782015)
Another great concurrency library in OCaml is the Async library. It's very convenient compared to Lwt. You can find here how it behaves with I/O operations https://realworldocaml.org/v1/en/html/concurrent-programming...
Re:A deep dive into Multicore OCaml garbage collector by fpoling on Sunday July 16, 2017 @05:53PM (#14784298)
Go story becomes rather bad the moment one wants to support IO cancellation without memory leaks in form of hanging forever go routines. As one cannot cancel them, a proper shutdown requires a lot of rather non-trivial code with thinks like sending channels over channels etc. Recently introduced Context was supposed to address it, but since one cannot use that to cancel a blocking system call, this is only helpful with high-level libraries that tries hard to work around that.

Lwt does have proper cancellation support. There are some pain points, but it is straightforward to work around them. The real problem with Ocaml is that there are 2 incompatible libraries for promise-based IO, Lwt and Async. That made it really hard for third-party libraries as now they need to support not one, but two frameworks...

Re:A deep dive into Multicore OCaml garbage collector by ice109 on Sunday July 16, 2017 @09:30AM (#14781900)
I whole heartedly agree with you on the convenience of go routines. I haven't been following the ocaml concurrency sorry; are they implementing a similar concurrency model?
Re:A deep dive into Multicore OCaml garbage collector by kcsrk on Sunday July 16, 2017 @09:45AM (#14781942)
Multicore OCaml comes with native support for concurrency through algebraic effect handlers which generalize common control flow abstractions such as exceptions, async/await, generators, non-determinism, backtracking. All of these mechanisms can be implemented directly in OCaml. [0] introduces the model, [1,2] has examples, and for further reading see [3].

[0]: http://kcsrk.info/ocaml/multicore/2015/05/20/effects-multico...

[1]: http://kcsrk.info/ocaml/multicore/effects/2015/05/27/more-ef...

[2]: https://github.com/kayceesrk/effects-examples

[3]: See recent pubs here http://kcsrk.info/.