Re:Astronomer to Data Scientist
by elchief on Saturday January 05, 2013 @02:31AM
(#5011994)
Some clarifications.
1. Java and Python are good languages for data mining, due to their libraries, though the others aren't great. C++ is good for finance.
2. Excel is not a statistical analysis package. "Just because you can drive a car with your feet doesn't make it a good idea" - Chris Rock. Excel is great for pivot tables, however.
3. Hadoop is not a distributed database, it is MapReduce plus a distributed file system. Hive is also not really a distributed database. It lets you write SQL that turns into mapreduce jobs.
Re:Astronomer to Data Scientist
by rjdagost on Saturday January 05, 2013 @10:07AM
(#5012793)
Excel is the most common statistical analysis package on the planet, bar-none. When you need to deliver code to people who are numerate but don't program then Excel is the best thing to use. Sometimes you just need to give someone (a manager) a tool where they can test out a bunch of different scenarios but you don't have time to make a polished application. Even for "big data" applications sometimes seeing the numbers in front of you in cells is quite useful.
Re:Astronomer to Data Scientist
by berkeleyjess on Saturday January 05, 2013 @12:56PM
(#5013366)
Thanks for the corrections. I have changed the post to reflect the Hadoop/Hive/MapReduce mistake. I am still new at this tech thing and obviously don't know all the terminology yet.
Re:Astronomer to Data Scientist
by noelwelsh on Saturday January 05, 2013 @03:36AM
(#5012097)
Looking from the other side, if I were hiring I wouldn't be too worried if the applicant knew this kind of stuff or not. It would be in their favour, but I'm far more interested in a strong mathematical background, intellectual horsepower, and personality. If they have these attributes (and any good researcher should) they should be able to learn the tech.
Re:Astronomer to Data Scientist
by 46Bit on Saturday January 05, 2013 @06:27AM
(#5012327)
If you've hired/been hired for a "Data Scientist" role recently, what are the main skills and accomplishments you look for? I could do with a better idea as to whether we're talking someone who's a statistics fan or a distinguished PhD expert.
Re:Astronomer to Data Scientist
by noelwelsh on Saturday January 05, 2013 @09:47AM
(#5012741)
At this point in time "Data Scientist" seems to be very loosely defined. I've seen everything from "must know Excel" to "must hold a PhD". You'll find lots of blog posts where people try to pin it down, but I don't think one definition will suffice. A more likely outcome is that we'll recognise specialisms within data science just like we have front-end, back-end, and other specialisms within programming.
1. Java and Python are good languages for data mining, due to their libraries, though the others aren't great. C++ is good for finance.
2. Excel is not a statistical analysis package. "Just because you can drive a car with your feet doesn't make it a good idea" - Chris Rock. Excel is great for pivot tables, however.
3. Hadoop is not a distributed database, it is MapReduce plus a distributed file system. Hive is also not really a distributed database. It lets you write SQL that turns into mapreduce jobs.
On the accuracy of statistical procedures in Microsoft Excel 2007
http://or.nps.edu/faculty/PaulSanchez/oa4333/handouts/Excel/...