Re:Job-Scoped Hadoop Clusters with Google Cloud
by cutler on Wednesday June 07, 2017 @02:43AM
(#14504166)
Can anyone tell me how, as a sole developer, it's possible to gain real-world experience with distributed Hadoop and Spark given the massive computing resources required? It just seems like a closed shop to me.
[ Parent
| on HN ]
Re:Job-Scoped Hadoop Clusters with Google Cloud
by teraflop on Wednesday June 07, 2017 @02:57AM
(#14504232)
Define "massive". You can learn all the most important aspects of the APIs and programming model on a single machine, either by running in "local" (non-distributed) mode or by running a few VMs to simulate a real cluster. And you can spin up a cluster of 100 machines on either AWS or GCE for about a dollar per hour.