I'm trying to make the switch from backend development to machine learning right now, and even though I have a supportive employer I've begun thinking about going back to school because the math involved seems steeper and harder to avoid.
Disclaimer: My math level is high school since I self-taught my way into software engineering.
You might be interested in the machine learning library I wrote for spaCy: https://github.com/explosion/thinc . The backward propagation can be understood quite simply with callbacks.
The main problem I had, and I think you're having, is that the notation around calculus is just hopeless. It's even worse when you're trying to map between the equations and backpropagation, because what you need is slightly different. I also find the notations for functions really bad.
The other thing you need to get used to are the ways linear algebra overload the operators. This is just data, not concepts. Personally I find it much easier to just work in the Einstein summation: https://obilaniu6266h16.wordpress.com/2016/02/04/einstein-su... . Once it's debugged, you can translate into faster calls to numpy.tensordot.
Don't go back to school. School's fucked. It'll take you three years to do about 200 hours of learning. Hiring a tutor would be vastly better if you need some help.
I can study on my own, and have been for stats/probability. But started asking myself, why not just get a degree and make it quicker. I'd have to leave my job for a number of years, but then I'd be able to study all day instead of nights/weekends.
You can only make this claim if you've done it.
But in terms of basic skills that are required to work in the field, the prerequisites aren't anything particularly demanding. When you look at most of what's published in the ML area, the bulk of what's required to understand it is contained in Calculus, Probability/Statistics and Linear Algebra. Maybe Topology for some things. But we're mostly not talking even anything as advanced as Abstract Algebra, Galois Theory, Group Theory, etc.
We are not talking about understanding what has been published in the ML area. We are talking about publishing your own, new ideas in peer-reviewed journals/conferences, and/or pushing state of the art results. That's what "doing research in AI" is all about.
Same applies to doing research in all those other fields you mentioned (calculus, statistics, group theory, etc). There's a big gap between people who can merely understand new ideas, and people who can come up with them.
if you do research in sorting/searching algorithms, even improving the performance of the best algorithm by a tiny bit would require quite a "special level of insight, or creativity, or whatever." Making a "BIG" advance would require genius level abilities, think a few dozens of people in the entire world.
Perhaps so, but we're not talking about research on sorting/searching algorithms. We're talking about AI. I would argue that the bar for publishing novel research is lower here, as the field is broader and correspondingly less developed (in areas).
I mean, right now, ML in particular is heavily based on empirical observation and experiment and lacks theory in many areas. So just trying some new combination of hyper-parameters that look interesting, achieving a "better than the latest published result" in some benchmark, and writing up a paper can constitute original research in this area. No, it's not the same as a major breakthrough in theoretical understanding, but then again not everything is. Which is at least part of my overall point.
It's probably also worth nothing that not all "research" happens in academic settings and not all researchers publish (or are driven to publish to the extent that academics are... especially tenure track folks). I'm thinking of corporate settings where simply achieving a useful result that helps the company make more money is considered a win.
All of that said, I'm open to the possibility that there are, indeed, fewer than 10k people who can do AI research, as I'm starting to wonder if there are fewer than 10k people who can write FizzBuzz successfully. But acquiring the basic tools necessary to do research in the AI/ML domain isn't so hard as to be inaccessible to a large number of people.
But...
I would argue that AI research (depending on how you define that. AI is a BIG field) is more demanding that "flavour of the month smartphone app" development. And in the past, I scoffed when I heard people say things like "70% of so-called developers can't complete FizzBuzz." Then I started interviewing a lot of candidates for developer roles at the $dayjob and whaddaya know... it seems that most of the candidates we get can't successfully code FizzBuzz. And that's a pretty low bar.
So I dunno... part of me believes there are plenty of people who could contribute to AI research, but part of me is shocked by how few people can even write the most trivial computer program.
Are there fewer than 10k people with a master's in the subject in the entire world?
I don't think having a Masters has much to do with anything. We get folks all the time with a Masters in C.S., even from relatively prestigious schools, and they can't even code up FizzBuzz.
Anyway, I wish I had a better answer for you, but recent experiences have me questioning just how many really competent people there are out there.