Re:How to Analyze Billions of Records per Second on a Single...
by smw on Wednesday July 11, 2018 @06:25AM
(#17505783)
No, the point was to be able to process workloads much larger than would fit in memory on a single machine.
[ Parent
| on HN ]
Re:How to Analyze Billions of Records per Second on a Single...
by mmt on Wednesday July 11, 2018 @02:46PM
(#17509277)
Citation needed! :)
Joking snark aside, I'm actually doubtful this is true. Specifically, I don't recall the impetus for Hadoop (or Google's original Map-Reduce, as described in the '04 paper) being an all-in-memory workload.
Despite it being repeatedly brought up in this sub-thread, I maintain that it's a niche use case and that disk-based data processing workloads are far more common.
ETA: Does anyone know of a canonical or early/initial document outlining the purpose, or at least design goals, of Hadoop?