News for hackers. Stuff that matters.
Re:Ask HN: What's the state of the job market in data scienc... by hardtke on Wednesday December 21, 2016 @05:27PM (#13233211)
I hire machine learning engineers and data scientists. In my opinion there is a great shortage of truly qualified machine learning engineers. A lot of people are entering the market with a general knowledge of machine learning tools. These people should be considered analysts or product data scientists. When it comes to people that can build machine learning systems that work at scale, they are very rarely available for hire and often are the subject of bidding wars by multiple companies. The key difference is whether the candidate truly understands the mathematical and statistical basis of machine learning, has the programming skills to execute their ideas, and is able to write code that can be used in large scale production systems and can be leveraged by others.
Re:Ask HN: What's the state of the job market in data scienc... by user5994461 on Wednesday December 21, 2016 @05:31PM (#13233231)
And that's why I think finance is smarter.

They've long understand that there are the finance analysts on the one hand and the software dev on the other. They get both and make them work together.

Looking for 5 rare skills in a single person is bound to disappointment: maths, statistics, programming, large scale systems, production.

Re:Ask HN: What's the state of the job market in data scienc... by hardtke on Wednesday December 21, 2016 @05:37PM (#13233275)
Companies that have a research division and a separate engineering team to implement the research ideas are rarely successful. Anyone engineer good enough to do the implementation will figure out they can build something better without the input of the researchers. Microsoft and Yahoo both had great research teams whose ideas rarely saw the light of day.
Re:Ask HN: What's the state of the job market in data scienc... by user5994461 on Wednesday December 21, 2016 @05:41PM (#13233301)
I didn't say to put them in two separate divisions. Put both guys in the same place, working closely with each other.

Any engineer will quickly figure out that he's out of his depth in the maths & statistics. Any mathematician will quickly figure out that he's out of his depth in the system building.

Re:Ask HN: What's the state of the job market in data scienc... by infinite8s on Tuesday December 27, 2016 @03:55PM (#13266106)
And this is why a lot of trading desks would couple a trader with a quant with a technologist.
Re:Ask HN: What's the state of the job market in data scienc... by runT1ME on Wednesday December 21, 2016 @05:33PM (#13233255)
> The key difference is whether the candidate truly understands the mathematical and statistical basis of machine learning

Can you elaborate on this, and at what level? Are you talking about a PhD level of understanding of cutting edge mathematics, or do you mean understand the basics, or somewhere in between?

Re:Ask HN: What's the state of the job market in data scienc... by shas3 on Wednesday December 21, 2016 @06:06PM (#13233450)
I think the following interview questions can help screen candidates (not an exhaustive list):

1. Asking a candidate to cast a non-standard problem of classification or estimation into a tractable optimization problem. (this is a very valuable skill that someone who has done good studies in numerical linear algebra/machine learning/stat/information theory/control systems/signal processing/math/etc. should be able to do)

2. Asking them to take an algorithm they have used and explain every step in deriving the algorithm. (It will help interviewer calibrate the level of learning in the interviewee. Also helps screen for indisciplined black-box users.)

3. Presenting challenging machine learning scenarios: using customized ensemble learning approaches, imbalanced data sets, noisy labels, multiple instances, different error metrics, etc. and seeing how interviewee approaches the problem from first principles (real world problems almost always involve some of these issues)

4. Testing their intuitions in "feature-engineering" for different types of data. (with the partial exception of cases where rigorous research/successful products show the utility of deep learning, one has to almost necessarily do a fair bit of feature-engineering)

Re:Ask HN: What's the state of the job market in data scienc... by ice109 on Friday December 23, 2016 @02:01PM (#13246553)
very good response
Re:Ask HN: What's the state of the job market in data scienc... by plafl on Wednesday December 21, 2016 @06:40PM (#13233610)
I usually ask about what the candidate has worked on. Or maybe the topic of their PhD thesis. I prefer that people explain to me what they know instead of having a list of questions about what I know.
Re:Ask HN: What's the state of the job market in data scienc... by hardtke on Wednesday December 21, 2016 @05:43PM (#13233312)
Somewhere in between. A simple question I use is "how do I decide whether to add a feature to a classification model?" Most candidate are fine until I bring up the topic of correlated features.
Re:Ask HN: What's the state of the job market in data scienc... by DrNuke on Wednesday December 21, 2016 @05:49PM (#13233344)
Are you hiring for finance? In Europe it still depends very much on the industry: some mean coding, some maths, some infrastructure, all want data preparation and in-depth stats. edit: other than understanding the business fundamentals, of course.
Re:Ask HN: What's the state of the job market in data scienc... by akvadrako on Wednesday December 21, 2016 @05:52PM (#13233362)
Do you mean like non-orthogonal dimensions?
Re:Ask HN: What's the state of the job market in data scienc... by photoJ on Wednesday December 21, 2016 @05:55PM (#13233384)
That's terrible.
Re:Ask HN: What's the state of the job market in data scienc... by notyourwork on Wednesday December 21, 2016 @05:48PM (#13233339)
My colleague focused on machine learning for his phd and this is a very accurate description. Talking the talk can come with experience, being able to understand the math and execute a new system at scale is where the separation in experience will start to surface.
Re:Ask HN: What's the state of the job market in data scienc... by IndianAstronaut on Thursday December 22, 2016 @04:24AM (#13236037)
Someone who can build models and someone who can scale it are 2 different people and professions. Data scientist vs data engineer.
Re:Ask HN: What's the state of the job market in data scienc... by gech on Wednesday December 21, 2016 @10:11PM (#13234462)
Whats the time from hire to a built live production system you would expect a truly qualified engineer to be able to achieve?