News for hackers. Stuff that matters.

A comment on Computing shortest distances incrementally with Spark. Parent comment

Re:Computing shortest distances incrementally with Spark by ignoramous on Tuesday June 16, 2015 @05:35PM (#9728850)
A better approach at the scale being discussed in the article would be to...?
[ Parent | on HN ]
Re:Computing shortest distances incrementally with Spark by karussell on Wednesday June 17, 2015 @03:51AM (#9730495)
What does 'scale' mean :) ? Solving graph problems are best done in-memory with a big machine (RAM!), otherwise you are an order of magnitude slower (at least) if you try to distribute. But if you really have no choice I would have a look if some of the graph databases has a good distributed model. My gut feeling tells me that even a bad approach there is faster than spark ...
[ Parent | on HN ]
Re:Computing shortest distances incrementally with Spark by ignoramous on Wednesday June 17, 2015 @11:12PM (#9736646)
I guess Titan DB would fit the bill: http://thinkaurelius.github.io/titan/

Facebook has blogged about scaling Apache Giraph to insane number of Vertices/Edges.

[ Parent | on HN ]