As with data science, the number of people that say they do "data science" and "machine learning" are in far greater supply than the number of opportunities.

Likewise, the number of people that are able to apply their domain knowledge to actually help their company's businesses/product or problems will always have work.

I've often used an analogy comparing this to cooking. Just because people say they know how to "cook" a dish, doesn't mean they'll be able to function or be effective in a restaurant or other type of business context kitchen. Yes, there's an element of this domain knowledge required, which are table stakes, but often the factors that distinguish success have more to do with understanding the context of the work and the surrounding business.

To update this for recent fads, learning to make sourdough bread does not mean one has learned how to run a bakery.

If you're able to be a pure researcher, or pure specialist, you're very lucky and working somewhere whey they can afford to have deep specializations.

The plethora of emerging tools and practices related to AI/ML "workflow" are related to the collective realization that training ML models is just one piece of the puzzle:

https://papers.nips.cc/paper/5656-hidden-technical-debt-in-m... (See: Figure 1)

But keep in mind, the VAST majority of those people have only toyed around with ML libraries or frameworks on the most trivial data. There's A LOT of script kiddies right now, that just copy/paste the same tutorials, maybe even publish yet another "How to classify MNIST dataset with NN / SVM / Naive Bayes / etc." on medium or whatever, just to come off as authority.

So while there's a lot more competition, the majority of these don't even meet the minimum standards to get called into interviews.

If you have solid work experience to show for, then that alone should differentiate you from the the masses. Again, a lot of these people have NEVER worked with anything relevant, and can only show you a bunch of tutorials they've followed. IF lucky, they've actually taken on more exciting datasets from sources like Kaggle, and tried to come up with something.

edit: Not to mention the data - I get extremely impressed when someone actually shows a project which involves them collecting data from the wild, wrangling it, and then doing meaningful work.

Too many beginners only work on neat / ready datasets. You'd be surprised how many candidates get completely blindsided when asked to do some trivial tasks on pre-processing data.

I have been doing ML engineering professionally for the 6 years and have been involved in about one hiring round per year for the last 4. If you filter resumes down to “has at least 1 year of professional experience on a deployed ML product”, then you’re lucky to have one resume out of 100, and that’s after HR has had their crack at the applicant pool. We still have to hire people who don’t really have much actual professional ML experience (which imo is fine). We invite literally anyone who has shipped ML code professionally for interviews, and we get ghosted a quarter of the time. It’s not like we pay below market either, it’s just that the demographic of people who already know what they’re doing can still afford to be really picky.

> Too many beginners only work on neat / ready datasets. You'd be surprised how many candidates get completely blindsided when asked to do some trivial tasks on pre-processing data.

I would argue that beginners only -work- on neat/ready datasets and lack the appreciation or understanding that most of the time the data does not already exist in a usable form, and sometimes not at all.

The process for making model training possible, and the process for applying models in a useful way are not taught in curriculum, tutorials, and generally not addressed in other literature (e.g. academic/research pubs)

"Amateurs and beginners focus on trying to get things right. Professionals focus on making sure things aren't/don't go wrong."

> minimum standards to get called into interviews.

I am a bit curious on what is the minimum standard? It does feel to me (an outsider that took 1 class of ML in college) that you need at least Master in ML to get intuition on the probabilistic and linear algebra theory behind ML concepts.

>I am a bit curious on what is the minimum standard? It does feel to me (an outsider that took 1 class of ML in college) that you need at least Master in ML to get intuition on the probabilistic and linear algebra theory behind ML concepts.

On the computer science side it makes sense to understand how to build a tree, not just know how to use one. In computer science you're learning how to write algorithms, not just use them. On the MLE or Machine Learning Software Engineer side, the same rule applies and knowing linear algebra as well as probability theory is helpful so you can write your own ML.

On the data science side you're rarely inventing new ML. What you want to specialize in is: 1) data mining using ML, 2) data cleaning, by knowing what kind of input ML needs, 3) feature engineering, also by knowing what kind of input ML needs, and 4) What kind of ML is ideal to choose, eg, due to the bias/variance trade off.

A lot of these boot camps, classes, and books teach the underlying structure of ML to become an MLE, though for some sort of reason they often advertise it as data science. I find this odd, because MLE pays better and is in higher demand.

On the data science side we rarely need to know these finer details. We just need to know the ML's characteristics so we know when it is the right tool for the job, similar to knowing how to use a data structure but not needing to know how to invent new data structures.

A data scientist needs to be a research specialist which is more of a phd than masters skill, so knowing the underlying math also doesn't matter as much because we know when it is necessary to research it. It's not that a data scientist can't know it, and many do know it out of hobby or classes, but it's far from mandatory. Knowing probability theory and how to digest a problem into multiple paths forward, like how to collect data, is far more valuable as a skill.

And finally, many data scientists barely know how to write code. They're a kind of analyst. I feel like the job title isn't sufficiently explained so software engineers make a lot of assumptions mixing up MLE with DS.

edit: Also, Linear Algebra isn't that bad and is an undergrad class. Probability theory is taught lightly in DS101 freshman year, but the first year of getting a masters probability theory is often taught again to a much more rigorous. This can get a bit harder, but if you understand the basics it's not bad.

As for Uber, the AI researchers that were laid off were not from the self driving division.

Google's freeze as far as I know is for all engineers ( only backfilling attrition is permitted), not limited to ML.

I personally feel this slump in the ML job market is only temporary and related to a temporary loss of risk taking appetite.

As for the legit vs non legit ML practitioners, I don't see how the situation is going to improve any time soon. Because it's not just random engineers fooling themselves by saying they are great at ML, there are entire companies new and old where there has been an increasing focus/pressure to sell AI powered $XYZ snake oil. They don't care who they hire, and most likely they don't even know who/how to hire when it comes to ML/AI, because the whole process right from the top is let's sell it first and wing it later

Also these hiring freezes may be due to covid, not loss of hope in ML. Google's AI residency program seems to be delayed but still going ahead, almost all the others were cancelled due to covid logistics (to my knowledge, at least FAIR, MS and Uber).

That doesn't detract from the article's point though - lots more interest in those programs than there are places.

I'd also add that the cuts to ML teams seem to reflect more of a freeze on R&D in general--totally normal in a downturn--and that ML just happens to be a feature in a lot of R&D at this point.

Most of these companies have substantial investments in ML that aren't research-focused, they're powering their flagship products. At this point, ETA prediction isn't a speculative experiment for Uber or Google Maps, just like RankBrain, Smart Compose, spam filtering, etc.

There’s a real lack of good engineering in machine learning and too much emphasis on using bleeding edge research.

There’s a lot of boring models and practices that get the job done for the 80%. That don’t require as much talent to do well. That scale, quick to develop, and can be implemented on prod with ease.

But there’s a problem in tech culture of reaching first for the most sophisticated ML, not the most practical.

So my perspective is that 2020:AI is as 2000:Internet. In 2000, the idea that all commerce is going to be on the Internet was obviously true, just as obviously true as the fact that all of the companies that IPOed at the time had no way of getting there (except for Amazon). AI one day is going to be the foundation of successful money-making products and services, but AlexNet and GPT-3 are not even stepping stones in that direction. Self-driving cars are a distraction like virtual worlds.

It's entirely possible, but also AI is older than the internet, and Google was the first big ML company, gaining their success as early as '99. It seems like ML has a longer adoption curve to it, moving slower.

Also, AI is anything cutting edge. The fill tool in Paint was once considered AI, so AI will continue on as long as we have new tech.

> The fill tool in Paint was once considered AI

This Wikipedia page has a lot of filling algorithms, and none of them looks like it's AI-based: https://en.wikipedia.org/wiki/Flood_fill

Can anyone explain me why the fill tool in Paint was considered AI? Has the definition of "AI" changed?

It would be AI by the standards of A*, maze solvers, and similar. It isn't ML and doesn't isn't related to that branch of AI.

Should graph search be considered AI? This gets us to the root of the issue, which is how do we consider something AI or not and to what extent has that definition changed as some problems have become trivial or commonplace. Machine learning is a bit easier since you can create much strong classifications around the need to teach, to feed it data with expected results, before it is useful to get a response (or otherwise give it a way to evaluate data it generates and then feeds itself). However there are a number of techniques that predate ML, or at least having fast enough computers to have worthwhile success at ML, and thus we are left with determining some way to objectively classify those as either AI or not.

It is technically not AI, but it used to be called AI.

AI today is the study of optimizing NP problems through a series of educated guesses.

> Has the definition of "AI" changed?

https://en.wikipedia.org/wiki/AI_effect

It would seem that the AI goalposts are always being moved.

They weren't saying the fill tool was an early form of what we think of as an "AI algorithm" today, but that the leading edge of machine intelligence / intuitive capability has changed, and it will continue to change.

Also you can probably implement fill with a graph search algorithm.

Yep, people forget that 'pagerank' was Google's secret sauce and it was literally ML. It looked at data to compute parameters of a model (edge probabilities) and rank a set of candidate results.

My first salary job was reverse engineering page rank. It was a lot of fun. /random

I'm having a hard time seeing PageRank '99 being ML just because it uses a lot of data and is linear algebra at the core. It's one large eigenvalue problem.

It's just because you're having a hard time seeing that most ML is linear algebra at the core.

No.

I mean it's true that ML uses linear algebra. But the difference between ML and PageRank is that ML is Machine Learning. The learning part (involving storing and updating based on new data) is different to statically calculating Pagerank using Map/Reduce and applying it as a ranking function.

Isn't updating parameters(in ML) same updating some ranking function parameter. In my opinion any algorithm that update its model parameter based on data is ML.

This is true, but in this case it was months between updates (in the early days of Google).

I take a pretty broad view of ML, but there certainly is a difference in degree if not principle.

Most machine learning in use today is “train once (and maybe finetune) and then deploy“, not online, continuous learning. I don’t think the frequency of model updates is generally a good indicator of whether or not something is considered machine learning.

More to the original point, nowadays when most people thick of machine learning they are thinking of deep neural networks, whereas Google‘s original pagerank was very simple and shallow by comparison. But they built an algorithm that allowed machines to learn what pages were high value and what pages were low value. If that seems simple by today’s standards, it’s evidence of the AI goal posts moving more than anything else.

I empathize. Been doing ML for a similar amount of time, and the amount of people who claim to know ML, but lack basics, abound. I don't think they are to blame (after all learning isn't bad, and everyone wants to capitalize on a market); we need to adapt to the fact that the definition of ML practitioner has considerably broadened: from someone who knows only to use vgg16 from PyTorch, without understanding the optimization, loss, alternative algorithms etc to someone with a much broader and deeper view of the area. Problems arise when people expect candidates of the latter type but instead end up running into a steady stream of candidates of the former type.

Hiring Managers probably need to get used to the fact the term "ML practitioner" alone doesn't mean much anymore, and they need to set expectations accordingly.

# of people who want to do "machine learning" > # of ML jobs > # of people good at ML

Yes, I was going to say. Taking courses in "ML" is a great way to get to write a hyped-up term on your resume. Actually understanding and being able to write code for statistical learning is something else entirely.

Edit: On the other hand, I think Carmack observed that as long as you get the sign right, you'll get something out of it, even if it won't learn very quickly. So maybe speed of implementation and learning is how skilled people will differentiate themselves?

Research tasks take a long time. If you're going off of speed, it's no longer data science it's software engineering. (Though, to be fair, with all the new software engineers becoming data scientists, the data science title is becoming more like software engineering.)

imho metaphysics and metalearning might be a better way to differentiate.

Right. There's room for research, and then there's room for commoditising existing research. I think the latter is where we currently have the most to gain from expanding our efforts in, and this was the context in which I made my comment.

I see now that I made that very unclear. Sorry!

No need to be sorry. I'm equally to blame for my own lack of communication skills. ^_^

There should be a law that require all ML and DS classes, books, and bootcamps start by manually labeling data.

Exactly why as a grad student I have decided to go into Quantative Finance instead of Data Science.

Literally anyone can become a data scientist, which is not a bad thing by all means. But for me, it makes it hard to justify me going to an elite French school and dealing with theoretical stuff (that will not be useful on a job, think VC dimension and all) for over a year.

Good luck!

I started with quant research and was successful at it before moving over to data science. It's a lot of fun! The technical challenges are more difficult too.

Part of it is the area. I'm in the SF/Bay Area. The other part is schooling as the industry is a bit more ivory. I got my first tech job when I was 17.

> Hopefully the industry matures. I've bumped into data scientists who don't know how to do feature engineering as well as ones who can't grok ML. It makes the whole industry look bad.

Agreed

I suspect that for 9 in 10 ML "experts" in the field there are corresponding 9 in 10 ML "jobs" in the field that are just hiring for the buzzword and seeing what happens

Have you ever interviewed applicants for software jobs? I think at least 9/10 applicants often have no idea how to code. They can write down some JavaScript, but don't know why.

>As someone who does ML for a living this hits. When switching companies in 2019 for the first time I had a hard time despite having 10+ years experience. Linkedin maxed out saying over 200 people had applied to every job I was looking at.

sure you're not just getting old? My experience is that once I got over 10+ years in experience in development people generally didn't want to hire me full-time, only consulting.

>Hopefully the industry matures. I've bumped into data scientists who don't know how to do feature engineering as well as ones who can't grok ML. It makes the whole industry look bad

This is what happens when a tech industry matures! Before it's mature and lucrative only the dedicated are into it, once it has matured there are layers of skillfulness and somewhere at the bottom the totally unskilled trying to get something.

I suppose you will be receiving an ML take home project for your next job shift.

>sure you're not just getting old? My experience is that once I got over 10+ years in experience in development people generally didn't want to hire me full-time, only consulting.

I don't know. I don't know how to break into consulting but I want to do it.

I've been through three IPOs in the last 11 years, and we didn't really hire consultants (except people out in the field to collect data sometimes, but that was contract employees), so I'm a bit out of the loop. I did everything wearing multiple hats, even going so far as to productionizing models I've made onto embedded, so there has been little reason to hire outside help.

I must have been half way awake. Not three IPOs. I went through one of those. I went through three acquisitions.

There's always a land rush of pretenders in any new sector of tech. Not sure where I heard it, but there's a definition of expert that I think a lot of people believe: "an expert is a person who knows one more thing than everyone else." It takes a while for experience to smoke the paper-thin experts out.

Honestly, I got my foot in the door and now get to work on pretty cool ML stuff as a DS.

I'm gonna stick it out for a few years (I like my job) and pivot to ML-PM right as there is a glut of technical people in the field.

In my experience, product/program managers that understand ML are a very rare breed, and often end up being the bottleneck when it comes to taking an ML model from research to users.

At the same time, becoming a PM too soon will be me being all talk no substance. Don't want that.

Good luck! ^_^

Google will happily hire and pay batshit crazy salary to any capable engineer with ML background

I'm interested to know more about your skill set. If you don't mind.

Ah sorry. I think you already mentioned timeseries classification.

np.

I've done quite a few out there things on previous projects. Some of the more extreme are:

- I invented a new kind of ML for quant work (day trading bot) that ended up being quite successful. It's not published anywhere for hopefully obvious reasons.

- I've written models that I've productionized to embedded. I also wrote a script that converts between languages, automating the process.

- I reverse engineered google's page rank tech (my first DS job actually) and ended up getting higher accuracy on website classification than a team of experts manually classifying web pages could do.

That sounds impressive!

I'm a student who just started a real life project (basic Visual Computing). Any practical tips of what I should look out for / focus on in the future?