Re:Spark Breaks Previous Large-Scale Sort Record
by gtrubetskoy on Friday October 10, 2014 @11:49PM
(#8441181)
I remember Nathan Marz saying that Storm is the most active project on Github about a year ago. ;)
Re:Spark Breaks Previous Large-Scale Sort Record
by nchammas on Saturday October 11, 2014 @04:23AM
(#8441611)
That may well have been true a year ago, but it is not true as of a few weeks ago [1], and hasn't been true since around Spring of this year.
As pointed out in the article multiple times, we are comparing with MR here. We are not comparing with Hadoop as an ecosystem. Spark plays nicely with Hadoop. As a matter of fact, this experiment ran on HDFS.
In terms of vibrant community, Spark is now the largest open source Big Data project by community/contributor count. More than 300 people have contributed code to the project.