Re:Infochimps cluster chef
by mrflip on Thursday June 24, 2010 @08:17PM
(#1459589)
Much easier to configure and install.
* You get Pig, Wukong and Dumbo out of the box. Don't learn Java hadoop, use one of those.
* Standardized setup which really helps if you have to seek IRC or other outside help
* Reasonable default parameters tuned for each of the EC2 instance sizes.
* Spinning up and tearing down a cluster is so much easier: experiment away. EC2-backed instances help this too.
Now of course you're trading the complexity of setting up Hadoop with the complexity of setting up chef and poolparty and EC2 -- but those are far less esoteric; and they either work or they don't.
* You get Pig, Wukong and Dumbo out of the box. Don't learn Java hadoop, use one of those. * Standardized setup which really helps if you have to seek IRC or other outside help * Reasonable default parameters tuned for each of the EC2 instance sizes. * Spinning up and tearing down a cluster is so much easier: experiment away. EC2-backed instances help this too.
Now of course you're trading the complexity of setting up Hadoop with the complexity of setting up chef and poolparty and EC2 -- but those are far less esoteric; and they either work or they don't.