Being a Data Scientist: My Experience and Toolset

Re:Being a Data Scientist: My Experience and Toolset by curiousgal on Tuesday January 17, 2017 @02:38PM (#13420429)

>stepping through complex applied maths functions manually in pen and paper.

We do that because:

A it helps us understand them better

B it teaches us how to think, the way Feynman said "Know how to solve every problem that has been solved". Granted, it seems pointless to work through what is easily accessible through machine BUT it teaches how to solve new problems. I wouldn't consider using NumPy or Matlab as the first step towards solving a new math problem.

It's like using Assembly vs using a higher level programming language.

[ Parent | on HN ]

Re:Being a Data Scientist: My Experience and Toolset by thinkr42 on Tuesday January 17, 2017 @02:50PM (#13420540)

Completely agree. There's a lot of nuance in these algorithms, they're not as cut and dry as simply calling a package method and oftentimes they aren't optimized to your use case. I work in Machine Learning, specifically on NLP, and it is really obvious when interviewing potential employees who knows what SVD means and who just know the NumPy function. Most "data scientists" I've interviewed fall in the latter category.

edit-This is of course completely anecdotal experience.

[ Parent | on HN ]

Re:Being a Data Scientist: My Experience and Toolset by lacampbell on Tuesday January 17, 2017 @04:07PM (#13421201)

I suppose my real question is - how many times do we need to do it? Once we have stepped through it by pen and paper once, or derived the result, how many times do we need to keep doing it? My experience in that mathematicians will do this again and again and again.

[ Parent | on HN ]