News for hackers. Stuff that matters.
Re:Spark: Open Source Superstar Rewrites Future of Big Data by hobbyist on Wednesday June 19, 2013 @07:15PM (#5909111)
Good question. I did read the spark paper, and one reason that I found for spark doing so much better than hadoop was that it avoids the unnecessary serialization, deserialization which hadoop just can not avoid. The RDD's as mentioned by @rxin, are in memory objects and thus do not require frequent serialization/deserialization when multiple operations are being applied to data.

There are no comments on this comment yet.