Re:On the Computational Complexity of MapReduce
by ihodes on Sunday October 05, 2014 @08:17PM
(#8413870)
Do you know of a group using Spark on petabytes (or even a terabyte(s)) of data?
[ Parent
| on HN ]
Re:On the Computational Complexity of MapReduce
by TallGuyShort on Sunday October 05, 2014 @11:17PM
(#8414175)
The scalability of Spark is certainly not as established as that of Hadoop. It's entirely anecdotal as I'm not in a position to share verifiable details, but I know of several installations that run Spark on several-hundred-node clusters. They're certainly well into terabytes, if not in the neighborhood of petabytes.
Re:On the Computational Complexity of MapReduce
by rxin on Monday October 06, 2014 @01:15AM
(#8414377)
There are several use cases that I know of. One public one is Taobao http://databricks.com/blog/2014/08/14/mining-graph-data-with...