News for hackers. Stuff that matters.
Re:Why scaling and parallelism remain hard even with new too... by rdtsc on Thursday August 11, 2016 @04:13AM (#12266836)
Not sure I know what you mean by "data parallelism". I wrote code not too long ago which had multiple processes reference the same binary blob. I have enough CPUs to ensure some processes were actually accessing it in parallel. In other case, used an ETS table to have multiple processes read and write data to and from it.

Or maybe to put it another way, if you think I am not already doing data parallelism, sell me on why I need data parallelism.

Re:Why scaling and parallelism remain hard even with new too... by catnaroek on Thursday August 11, 2016 @06:06AM (#12267102)
> reference the same binary blob

Is there some way to share data whose structure is actually understood and checked (statically or dynamically) by the language?

> to have multiple processes read and write data to and from it.

How did you coordinate those reads and writes? A writer not having exclusive access to what it's writing to, at the time of the writing... sounds... very bad.

> sell me on why I need data parallelism.

Just to name one example: machine learning. If you're feeding real-time data to a ML algorithm, chances are you'll have to perform multiple independent calculations that use the same data as input. To speed up those calculations, you need to perform them in parallel.