Elemental: C++ library for distributed-memory linear algebra and optimization

Re:Elemental: C++ library for distributed-memory linear alge... by math_and_stuff on Monday December 28, 2015 @04:26PM (#10803489)

I would say that the goals align much more than the current approaches, which are somewhat orthogonal. My focus in developing Elemental has been on expanding from the scope of libraries like ScaLAPACK (which focused on distributed dense linear algebra) into more general algorithms (e.g., Interior Point Methods and lattice reduction) and modern programming practices (using templates to support arbitrary-precision versions of the above).

Arguably the biggest weakness of Elemental is its lack of focus on dynamic scheduling at an intranodal level, though this can to a large degree be pushed into a local BLAS layer. One of the big items on my wish list would be proper integration of intranodal dynamic scheduling with the current high-performance internodal static scheduling.

The current TensorFlow releases seem to be primarily focused on intranodal dynamic scheduling; there has been a substantial amount of work in this area from the PLASMA [1] and (less recently) SuperMatrix [2] projects.

[1] http://icl.cs.utk.edu/plasma/ [2] http://www.cs.utexas.edu/users/flame/pubs/SuperMatrixTR.pdf

[ Parent | on HN ]

Re:Elemental: C++ library for distributed-memory linear alge... by Gtifn on Monday December 28, 2015 @09:31PM (#10804818)

How about dask's distrubted array? Do you know how the architecture and goals compare to Elemental?

[ Parent | on HN ]

Re:Elemental: C++ library for distributed-memory linear alge... by math_and_stuff on Monday December 28, 2015 @10:38PM (#10805049)

Distributed [1] is very new and seems to have similar core architectural goals as TensorFlow. But perhaps I'm being too politically correct: both make use of very course-grain parallelism relative to a typical distributed-memory linear algebra library (e.g., the current communication mechanisms of both are likely to be too course-grain to efficiently support distributed dense matrix inversion or eigensolvers; not that this is likely to be a design goal of either).

[1] http://matthewrocklin.com/blog/work/2015/06/23/Distributed/

[ Parent | on HN ]

Re:Elemental: C++ library for distributed-memory linear alge... by Gtifn on Tuesday December 29, 2015 @12:11PM (#10807641)

I think distributed's scheduler has seen much iteration since then and is now based on tornado. I wonder if that changes your assessment.

Also what do you think of Julia's native distributed capabilities and this library here: https://github.com/shashi/ComputeFramework.jl

[ Parent | on HN ]