Re:Ask HN: What are the best tools for analyzing large bodie...
by bra-ket on Thursday June 18, 2015 @07:13AM
(#9737835)
Apache Spark
[ Parent
| on HN ]
Re:Ask HN: What are the best tools for analyzing large bodie...
by johan_stenberg on Thursday June 18, 2015 @07:40AM
(#9737919)
I would not use Apache Spark when the amount of data is this little. I have myself done this mistake when processing only a few gigabytes.
Take a look at this: http://aadrake.com/command-line-tools-can-be-235x-faster-tha...
IMHO I think people abuse Spark and I would be truly impressed if anybody could write a Spark program faster then just a regular Scala program for processing this.