My sense is that the original sin here is conflating data science with machine learning.

A good data scientist might choose to use machine learning to accomplish their job. Or they might find that classical statistical inference is the better tool for the task at hand. A good data scientist, having built this model, might choose to put it into production. Or they might find that a simple if-statement could do the job almost as effectively but not nearly as expensively. A good data scientist, having decided to productionize a model, will also provide some information about how it might break down - for example, describing shifts in customer behavior, or changes in how some input signal is generated, or feedback effects that might invalidate the model.

OTOH, if your job has been framed in terms of cutting-edge machine learning, then you may well know - at a gut level, if not consciously - that your job is basically just a pissing match to see who can deploy the most bleeding-edge or expensive technology the fastest. It's like the modern hospital childbirth scene in Monty Python's The Meaning of Life, where the doctor is more interested in showing off the machine that goes, "ping!" in order to impress the other doctors than he is in paying attention to the mother.

This matches my experience exactly.

About 5 years ago I was coming out of academia with a PhD in chemistry wanting to get into tech. Someone pointed me toward data science and I was immediately pulled in by the potential for deep insights and working with interesting data sets. After getting into an industry research position, I was quickly disillusioned by the talk from our leadership of how we needed to become an "AI company" and discovered that was I really wanted to be doing was classic algorithm development, not data science.

Now, I've see so much poorly implemented, unnecessary machine learning applied to problems that didn't need it that I first assume that any machine learning project is a bad technical decision until proven otherwise. I've happily moved into an engineering role building interesting pipelines.

There’s people who consider classical inference and the like to be machine learning just as much as neural nets are. I like that perspective.

There are some things, like OLS and logistic regression, that are commonly used for both purposes. But there's a sort of moral distinction between machine learning and statistical inference, driven by whether you consider your key deliverable to be y-hat or beta-hat, that ends up having implications.

For example, I can get pretty preoccupied with multicollinearity or heteroskedasticity when I'm wearing my statistician hat, while they barely qualify as passing diversions when I'm wearing my machine learning engineer hat. If I'm doing ML, I'll happily deliberately bias the model. That would be anathema if I were doing statistical inference.

Oh gotcha. That’s an interesting way to draw the line and I appreciate the distinction.

And this is a good thing!

To be fair, I started to understand why developers gave out about bootcamp grads lacking a foundation when the bootcamps came for my discipline (data science).

The PhD fetish is pretty mental (even though I have one), as it's really not necessary.

Additionally, everyone thinks they need researchers, when they really, really don't.

Having worked with researchy vs more product/business driven teams, I found that the best results came when a researchy person took the time to understand the product domain, but many of them believe they're too good for business (in which case you should head back to academia).

What you actually need from an ML/Data Science person:

- Experience with data cleaning (this is most of the gig)

- A solid understanding of linear and logistic regression, along with cross-validation

- Some reasonable coding skills (in both R and Python, with a side of SQL).

That's it. Pretty much everything else can be taught, given the above prerequisites.

But it's tricky for hiring managers/companies as they don't know who to hire, so they end up over-indexing on bullshitters, due to the confidence, leading to lots of nonsese.

And finally, deep learning is good in some scenarios and not in others, so anyone who's just a deep learning developer is not going to be useful to most companies.

Just an anecdote but if you go to most baseball data departments, where there is real competition between teams, you don't just have PHds. You have people with undergrads/domain knowledge, and people with PHds.

This isn't to say that PHd knowledge isn't valuable but if you look at firms in finance that have had success with data i.e. RenTech, they hire very smart people with PHds but it isn't only the PHd. You need someone who has the knowledge AND someone who has common sense/can get results. That is very hard to do correctly (and yes, some people who come from academia literally do not want anything to do with business...it is like the devs who come from a CS PHd and insist on using complicated algo and data structure everywhere, optimising every line, etc.).

*PhD or Ph.D.

It's an abbreviation of the Latin "philosophiae doctor" == "doctor of philosophy".

Getting one is as much about persistence as intelligence, and makes you very knowledgeable about one narrow area, so what you say makes sense. Academic researchers usually branch out into other fields and subfields as well, but straight out of a PhD, narrow and deep is what you tend to get.

I worked in a place full of deep learning PhDs, and you'd have people trying to apply reinforcement learning to problems that had known mathematical solutions, and integer programming problems.

I don't think the issue is just that companies hire people who are awful at ML, it's also that people are trying to shoehorn deep learning into everything, even when it currently has nothing to offer and we have better solutions already. IMHO, we're producing too many deep learning PhDs.

This is just my general sense, as a very non-expert with more experience of doing than theory...but the benefit is someone knowing the theory AND being able to translate that into revenue.

I think most people view the hard part as doing the PHd, and so lots of people value that experience, and because they have that experience you have this endowment effect: wow, that PHd was hard, I must do very hard and complex things.

To give you an example: Man Group. They are a huge quant hedge fund, in fact they were one of the first big quant funds. Now, they even have their own program at Oxford University that they hire out of...have you heard of them? Most people haven't. Their performance is mostly terrible, and despite being decades ahead of everyone their returns were never very good (they did well at the start because they had a few exceptional employees, who then went elsewhere...David Harding was one). The issue isn't PHds, they have many of them, the issue is having that knowledge AND being able to convert it.

I think this is really hard to grasp because most people expect problems to yield instantly to ML but, in most cases, they don't and other people have done valuable work with non-ML stuff that should be built on but isn't because domain knowledge or common sense is often lacking.

A similar thing is people who come out of CS, and don't know how to program. They know a bit but they don't know how to use Git, they don't know how to write code others can read, etc.

The Man Group has had respectable returns, especially during Coronavirus. Nothing amazing, but certainly not terrible. Regardless, there's more to the picture: Sharpe ratio, vol, correlation to the market, etc

That isn't the case. First, I was talking about multi-decade, not how have they done in the last few hours. Second, their long-term returns haven't been good. They lagged the largest funds (largely because their strategy has mostly been naive trend-following). Third, you are correct that their marketing machine has sprung into action recently. But how much do you know about what trades they are making? If you were around pre-08, you may be familiar with the turn they have made recently (i.e. diving head first into liquidity premium trades with poor reasoning, no fundamental knowledge).

And again, the key point was: they have had this institute for how long? Decade plus? Are they a leading quant fund? No. Are they in the top 10? No. Are they doing anything particularly inventive? See returns. No.

How is this any different to developers who insist on using some shiny new web framework, micro service spaghetti and kubernates overkill infrastructure for their silly little CRUD app?

I don't think it is any different. Overvaluing the latest hotness is extremely common in the tech industry and is one of my least favorite parts of it.

Unfortunately, this is where the incentives of the company and that of the employee diverges. For the employee, if they choose some simpler, appropriate model or solution to the problem, they will not be able to get that next DL job. Especially early in their career. I cannot bring myself to do resume driven development, but I understand why people do it.

But you probably don't need a DL job. As my dad always said, as long as you make them/save them money, they'll never fire you.

I know that I (as a DS Lead/Manager) would hire someone who uses an appropriate solution to a business problem above someone who has an intricate knowledge of applying PyTorch to inappropriate problems.

But maybe I'm in a minority here.

It sounds so painful as someone all-in this area. But I have to agree on a task of overdoing with fancy models. Nevertheless, the most common ML algo in industry is still linear regression along with boostraping.

It's just a process of exploration, people trying out ideas to see what works. Over time, with sharing of results, we will gradually discover more nuanced approaches, but the exploration phase is necessary in order to map a path forward and train a generation of ML engineers and PM's who don't have seniors to learn from.

Of course it sucks on the short term, but there is zero chance the field will be abandoned. It has enough uses already.

But the exploration should be led by the data scientists, not by the suits who think they need a specific type of model.