Data Science Is America’s Hottest Job

Re:Data Science Is America’s Hottest Job by mikert5671 on Friday May 18, 2018 @12:12PM (#17102585)

I actually think that machine learning can be much harder than you are claiming here, and that your inexperience is showing when you make such claims. Sure a toy problem from some dataset in a book or pulled off the internet is easy. So is implementing a k-means algorithm. But go to a large corporation with 100 datasets, all with 100 fields each, each having significant seasonal bias (among other biases), and build something that lasts, works, is clean, and is better than someone else can build. You need to convince them to trust you, maybe you can write an algorithm that works, but the business people cant understand that algorithm, and so they have to trust your word.

[ Parent | on HN ]

Re:Data Science Is America’s Hottest Job by nightski on Friday May 18, 2018 @12:36PM (#17102792)

Often this comes down to the ability to communicate, work with people, and understanding real world business. In addition, often the more simple a learning method used the better. I actually find most businesses needs can be solved by straightforward analytical methods such as traditional statistics. Very few situations require actual machine learning and it is often misapplied.

In fact, having a PhD does not tell you a whole lot about these skills.

More advanced modern techniques such as deep neural networks, reinforcement learning, etc.. are all extremely proficient at certain niche problems but these do not come up nearly as often in a business context.

This is why I don't advertise myself as a machine learning engineer. Rather, I am a business consultant that knows when, and when not to utilize machine learning methods.

[ Parent | on HN ]

Re:Data Science Is America’s Hottest Job by randcraw on Friday May 18, 2018 @05:56PM (#17105278)

So true. Missing from university textbooks is the fact that real world data is often dirty as hell, inconsistent, distorted, biased, incomplete, and/or just plain invalid (measured the wrong source or too imprecisely to be useful).

Unless your DS group is big enough to warrant hiring data engineers / cleaners, as a data scientist it'll be your job not only to eventually choose the algorithm, but foremost, to confirm that the data is sufficient to serve the intended purpose of mining it, ideally before you waste a lot of time curating it or paying for a raw data dump you can't use.

[ Parent | on HN ]