News for hackers. Stuff that matters.
Re:Playing with 80M Amazon Product Review Ratings Using Apac... by minimaxir on Thursday January 19, 2017 @08:48AM (#13435692)
Essentially the same Spark caveats of lazy evaluation and immutability of caches: neither are a big deal on small datasets, but making a mistake on either on a large dataset can result in a lot of lost time or confusion.

Then there are the massive shuffle read/writes that result in 50GB i/o which are not great for SSDs.

There are no comments on this comment yet.