Re:Accidental Scientist Hawks ‘Online Marketplace for Brains’
by lurker17 on Tuesday December 13, 2011 @06:25PM
(#3349847)
I think you'll find that Hadoop is often used in preparing the data into bite-size chunks for use in Kaggle contests.
Also, sometimes the contest winners do win by fetching more data related data than Kaggle provides. This happened in the Wikipedia participating-prediction contest, for example.
Yes I've noticed this as well. People are often asking me for my opinion about Big Data, treating it as a synonym for Data Science. Since the journalist at Wired had recently written a couple of articles about Hadoop, I took this opportunity to explain to him how data size and data value are not necessarily directly related. I think he did a great job in the piece of getting this point across.