Re:Data Scientist: The Sexiest Job of the 21st Century
by keefe on Thursday September 20, 2012 @10:05AM
(#4548575)
I doubt it, simply because it's so easy to find big datasets to work on. It doesn't have to be professional, that's the nice thing about a data driven profession.
Check out http://www.kdnuggets.com/ for links to large data sets to work on and there are also some on amazon.
Also, yes you can certainly run hadoop on a single instance, but once you get into "real big" sizes you'll need a cluster to demonstrate expertise, be it on your local machines at your house or on a set of VPS or EC2 or whatever.
Or, I guess programmers and engineers could start using the big data tools even though they are not needed. Has anyone ran Hadoop on a single (multi-core) machine for this purpose?