Your first model should always be something that predicts a constant value, or maybe in really complicated cases something like a linear/logistic regression. Then you have a baseline to compare more advanced approaches to. But in order to understand how to use linear regression well, you need to understand how it works in the first place.
Also experiment structure, sampling design, hypothesis testing, etc. will tell you a lot about what conclusions you can and cannot draw from observational data, which is what a lot of ML is about.
This example: > It's full of people (you can see a lot of analyses on Kaggle) that "do machine learning" but make very silly mistakes (e.g. turn categorical data into a float and use it as a continuous variable when training a model).
Doesn't seem related to stats or probability at all to me. Just critical thinking skills
But if anyone wants to become an expert into ML/DS, learning statistics and probability is fundamental. Books like A First Course in Probability, Introduction to Statistical Learning and Elements of Statistical Learning, to name a few, are very important.
A lot of the mistakes done in practice are based on a lack of understanding in sampling techniques, how statistical metrics can be misleading and so on.
First thing I learned in statistics is the difference between quantitative and qualitative information. If someone knows this before hopping into Kaggle, they know that categorical features can't be used as continuous features.
Though for applied papers that's sometimes enough.
But heed Larry Wasserman's advice: "Using fancy tools like neural nets, boosting, and support vector machines without understanding basic statistics is like doing brain surgery before knowing how to use a band-aid."
For the 101 level, I agree; I’d say you need a good understanding of basic probs and stats, rather than a vague understanding of advanced topics.
If you can spot when somebody says something about a dataset, the assertion is true only if the data is normally distributed, but there’s no checking about the actual distribution, you’re probably good to go.
One clarification: you don’t need an extreme understanding of stats, probability, linear algebra, imho. If you already took college level classes, you’re likely to be fine.
So my point is, if it isn't making sense the way you are being taught, go explore other avenues. There's no way these machine learning algorithms would have made sense to me as a 20-something undergrad, but as a 40-something who can explore them via software rather than a whiteboard, they really aren't that complicated to get started.
[1] https://www.youtube.com/watch?v=j9WZyLZCBzs&list=PLUl4u3cNGP...
1) Get some statistics/probability basics. It's full of people (you can see a lot of analyses on Kaggle) that "do machine learning" but make very silly mistakes (e.g. turn categorical data into a float and use it as a continuous variable when training a model).
2) take a look at traditional machine learning approaches. Nowadays you're swamped by DL (a lot of good suggestions on this thread, I won't chime in), and you miss the fact that, sometimes, a simple decision tree, or dimensionality reduction approaches (e.g. PCA or ICA) can yield an incredible value in a very short time on huge datasets.
I had written a fairly short post about it when I finished my georgia tech path https://www.franzoni.eu/machine-learning-a-sound-primer/
3) It can take a lot of time to become effective in ML, effective as in, what you _manually create_ is as effective as picking an existing trained model, fine tune it, and use it. This can be frustrating: low hanging fruits are pretty powerful and you don't need to understand a lot about ML algorithms to pick them up.
4) Consider MOOCs or online classes. I took Georgia Tech OMSCS, I can vouch for it and some classes force you to be a data scientist and read papers as well, and you can have "real world" recognition and discuss with your peers, which is useful!