Re:Uncomplicate: Number crunching in Clojure
by agibsonccc on Thursday May 04, 2017 @12:47AM
(#14262525)
Right that's what I was talking about though. I mentioned spark and flink in my original comment.
Spark makes different assumptions about the computers than what MPI does. "HPC" is 99% of the time some variant of MPI with some sort of special hardware interconnect like infiniband.
It's just in general a different software stack with different assumptions.
Like I said- I work on gpu stuff for the JVM as well, the assumption is different with how the cluster is expected to be setup though.
Spark makes different assumptions about the computers than what MPI does. "HPC" is 99% of the time some variant of MPI with some sort of special hardware interconnect like infiniband.
It's just in general a different software stack with different assumptions.
Like I said- I work on gpu stuff for the JVM as well, the assumption is different with how the cluster is expected to be setup though.