The idea has been to use the naive (in the sense of being distribution agnostic) batched interface to filling/querying Elemental's distributed data structures to interface with Julia's `DArray`. This essentially boils down to one all-to-all communication of the row, column, and values of each update and should have a runtime that looks like O(n^2/p) + O(p) for n x n matrices over p processes. Ideally one performs such redistributions as few times as possible if wanting to use routines that only require quadratic work or less (such as matrix-vector multiplication); for dense factorizations the overhead should be reasonably ignorable.

PETSc has an Elemental interface for doing dense linear algebra. Can't do much better for an endorsement than that.

I'm definitely happy that the PETSc developers have put up with Elemental being such a moving target (it has expanded from dense factorizations to sparse-direct, Interior Point Methods, and now some number theory), as there have been substantial growing pains. Barry Smith has also been nice enough to poke me any time Elemental isn't valgrind clean.