News for hackers. Stuff that matters.
Re:Bagel: Large-scale graph processing on Spark by HairyFotr on Wednesday June 22, 2011 @05:00AM (#2682328)
Is Spark really as fast in real use, as the graph on the site claims? I don't think I've ever seen anything beat Hadoop by that margin before.
Re:Bagel: Large-scale graph processing on Spark by danjo on Thursday June 23, 2011 @02:24AM (#2686894)
We currently use Spark for generating reports that scan the same data set multiple times. We got a 20-30x performance gain over hadoop+hive. Spark's ability to keep data sets in memory works very well for our use case.