Re:Joining a billion rows 20x faster than Apache Spark
by edw on Thursday February 09, 2017 @05:23PM
(#13611589)
I think his point is that bloated, over-engineered Big Data systems—whether batch or streaming—are overkill for the vast majority of problems.
[ Parent
| on HN ]
Re:Joining a billion rows 20x faster than Apache Spark
by placeybordeaux on Thursday February 09, 2017 @05:50PM
(#13611775)
There are just many points that don't really apply to stuff like spark or tez that runs on YARN:
ex: Hadoop << SQL, Python Scripts
I completely agree with
Mapreduce << SQL, Python Scripts
I do a lot of my processing on sparkSQL and through RDD transformations as opposed to Mapreduce limiting, slow KV style processing.