Re:Ask HN: What's the state of the job market in data scienc...
by rcpt on Thursday December 22, 2016 @10:41AM
(#13237952)
> a couple lines of code using SciKit Learn and all the default parameters is enough
Except for when it's not - those are the cases you're hired for.
Re:Ask HN: What's the state of the job market in data scienc...
by apohn on Thursday December 22, 2016 @12:23PM
(#13238723)
There are also no shortage of "Data Science" projects that are basically just showing off (e.g. somebody in a large org trying to get recognition for being advanced) or using an algorithm to justify a decision that somebody had already decided to make. In this case, it's okay to solve the problems with a few lines of code and a nice presentation.
But if you have an important data driven decision that is going to cost you a lot of resources (money, people, time, etc), you want the analyst to be able to speak to what the data and/or model is telling you and why. When you have this type of problem and it's "solved" by a few lines of SciKit Learn, you should be prepared to have a really bad day at some point in the future. This is the type of Data Scientist that draws the ire of people who do analysis with more depth.
Re:Ask HN: What's the state of the job market in data scienc...
by DrNuke on Thursday December 22, 2016 @01:17PM
(#13239148)
Let's be blunt: firms needing real data science know where to go and find real data scientists. Anything less real, on both sides, is matter of opinion and here it comes free market.
I'm not coming from a CS background and don't purport to know absolutely all of the details of all the mathematics and theory behind many of the machine learning algorithms that I use. I try my best daily to expand my knowledge, understand the algorithms, and apply them appropriately. I would hope that any company who is looking for someone who has a PhD-in-CS-or-ML could weed someone with lesser knowledge out during the interview process.
With that being said, a couple lines of code using SciKit Learn and all the default parameters is enough to impress many non-tech companies that are looking for a way to use 'predictive' in their marketing materials. And they pay very well for it. I get the feeling that provokes the ire of people who think those types of basic implementations belong to the traditional label of 'data analyst'.
For what it's worth, I work with data sets that aren't quite large enough to justify anything more than Python, Pandas, SKLearn, Luigi pipeline, and PySpark. The vast majority of my time is spent cleaning the data and generating features, must less on the hyperparameter tuning, model training side itself.
Anyways, I think I'm rambling a bit.
I just want to say that I LOVE this job, whatever the label is, or whatever the hype surrounding the label is, and I hope it's around for a while before it's automated...