News for hackers. Stuff that matters.
Re:Infochimps cluster chef by hooande on Thursday June 24, 2010 @04:16AM (#1457589)
From my understanding, the biggest barrier to using hadoop/cassandra is that they are very difficult to install. Unless you have a lot of experience with similar technologies, the time spent learning new concepts isn't worth it for most startups.

Simple question: Does chef make it any easier to install/configure hadoop and cassandra? Or does this make it easier to deploy them once you already have things set up?

Re:Infochimps cluster chef by mrflip on Thursday June 24, 2010 @08:17PM (#1459589)
Much easier to configure and install.

* You get Pig, Wukong and Dumbo out of the box. Don't learn Java hadoop, use one of those. * Standardized setup which really helps if you have to seek IRC or other outside help * Reasonable default parameters tuned for each of the EC2 instance sizes. * Spinning up and tearing down a cluster is so much easier: experiment away. EC2-backed instances help this too.

Now of course you're trading the complexity of setting up Hadoop with the complexity of setting up chef and poolparty and EC2 -- but those are far less esoteric; and they either work or they don't.