Re:Uncomplicate: Number crunching in Clojure
by agibsonccc on Thursday May 04, 2017 @12:47AM
(#14262525)
Right that's what I was talking about though. I mentioned spark and flink in my original comment.
Spark makes different assumptions about the computers than what MPI does. "HPC" is 99% of the time some variant of MPI with some sort of special hardware interconnect like infiniband.
It's just in general a different software stack with different assumptions.
Like I said- I work on gpu stuff for the JVM as well, the assumption is different with how the cluster is expected to be setup though.
Stuff like: http://spark.apache.org/news/spark-wins-daytona-gray-sort-10...