From a cursory look the linked page covers a broad range of subjects necessary to understand the last 65 years of AI research, not just the 9 years since the deep learning boom that most people are familiar with.

That you know many graduate students with published papers in major conferences in machine learning who don't have a background in AI is ...unfortunate? I guess?

But I think your question is more along the lines of "why do I need Peano arithmetic to build an image classifier?" (or any machine learning application). To be clear, you don't. But without knowledge of some fundamental concepts in mathematics and the theory of computation, you won't understand the power and limitations of any system you may be capable of buidling. As a lay person, that's by the by. As a machine learning researcher, that'd be disastrous.

Unfortunately, as the OP points out, such lack of deep background in AI research is common today, and probably responsible for the absolute quagmire in which machine learning research finds itself.

>ah these parameters/network layout did this, here's a guess as to why

Or just mindlessly gluing stuff to solve a particular problem

>we took x model and trained it to recognise cats

then yes, you probably do not need much mathematical knowledge. However if you are researching fundamentals, neural tangent kernel, neural odes etc, then you will probably be pretty decent at mathematics. You will also probably have been exposed to Peano at some stage in your life, even if you don't remember it. Its just good background mathematical character development.

I would say even if you are more of an applied person, being exposed to the fundamentals probably helps shape your thinking. Yes you can still be a good productive programmer, but maybe you will have something missing. The person with a broader background may be quicker at solving a new problem, they have a bit more to draw on.

At the risk of being overly presumptuous, the opinion you give sounds like that of someone who does research in such an area. That's what it sounds like to me, at least, because I once argued similarly while I was working from a more mathematical side about the value of understanding things rigorously. And I would issue statements like "well, sure if you just want to put things into PyTorch, _sure_, but that's not _real_ math."

Unfortunately for us (or at least me, because why did I spend so much time learning this stuff?), many top ML or AI researchers have only a vague understanding of fundamentals, and I vehemently disagree that you need rigorous exposure to mathematics to contribute to ML or AI.

My experience is quite the opposite. Lots of people in mathematics / academia are asleep at the wheel. Well, they were at the wheel fifty years ago, and some departments have caught up, but overall mathematical rigor is not what made the field go forward. (D. Donoho's "50 years of data science" seems relevant).

If you are adamant that Peano axioms in particular are necessary, why doesn't your argument that they are necessary extend to things like the axiom of choice? You know what they say: "the axiom of choice is clearly true, the well-ordering principle is clearly false, and who knows about Zorn's lemma?" Or, if the introduction to Folland's Real Analysis is to be believed, if your research project depends on the validity of the axiom of choice -- choose another research project.

All this to say your post makes me recognize a sentiment I once had: mathematical rigor is important. That depends. I realize that more often than not I made that argument to convince myself such knowledge is relevant -- and that creates a dangerous blind spot (at least for an academic who wants to be useful to industry).

But that depends on what you mean by "contribute". Machine learning research has turned into a circus with clowns and trained donkeys and the majority of the "contributions" suffer heavily from what John McCarthy called the "look ma, no hands disease" of AI:

Much work in AI has the ``look ma, no hands'' disease. Someone programs a computer to do something no computer has done before and writes a paper pointing out that the computer did it. The paper is not directed to the identification and study of intellectual mechanisms and often contains no coherent account of how the program works at all[1].

Yes, anyone can contribute to that kind of thing without any understanding of what they're doing. Which is exactly what's happening. You say that "many top ML or AI researchers have only a vague understanding of fundamentals" matter-of-factly but while it certainly is fact, it's a fact that should ring every single alarm bell.

The progress we saw in deep learning at the start of the decade certainly didn't come from researchers with "only a vague understanding of fundamentals"! People like Hinton, LeCun, Schmidhuber and Bengio have a deep background not only in computer science and AI but in other disciplines also (Hinton was trained in psychology, if memory serves). Why should we expect any progress to come from people with a "vague understanding" of the fundamentals of their very field of knowledge? In what historical circumstance was knowlege enriched by ignorance?

______________

[1] http://www-formal.stanford.edu/jmc/reviews/lighthill/lighthi...

"These are mathematics necessary for AI, as in "Artificial intelligence", not as in "machine learning"."

Calculus and linear algebra are a tad older than that.

> isn't relevant today as determined by the community itself?

Which community? NeurIPS/ICML/ICLR/et al.? That's only a subset of the AI community.

Also, I'm not really sure that "pure ML/DL" is even a particularly in-demand field of research expertise right now; we've been absolutely saturated with supply for at least junior phd students in that subfield for close to an academic generation by now... whereas there is now a genuine drought of folks who speak "both languages" in AI. I think these days you really need a second field in order to not be in a horrendously bleak over-saturated labor market. E.g., systems and ML? God, please yes. "Symbolic AI" tradition and ML/DL? Yup, lots going on there right now. But pure NeurIPS et al. style work? You've gotta the best in a very, very, very, very crowded room.

From my point of view and much like you say, the interesting, groundbreaking work has moved outside strict deep learning research. I mean, I sure would think so, but here's the website of the International Joint Conference on Learning and Reasoning, that brings together a bunch of disparate neurosymbolic and symbolic machine learning communities for the first time:

http://lr2020.iit.demokritos.gr/

This is an active field of research with plenty of space for new entrants and full of intersting problems to solve and virgin territory to be the first to explore. I'm hoping we'll soon see an influx of eager and knowledgeable new graduates disappointed with the state of machine learning research and willing to do the real hard work that needs to be done for progress to begin again.

lol maybe if you want to go into academia but, speaking from experience, FAANG is paying handsomely for this skillset.

Edit: as a personal recommendation, try not to start your comments with "lol". It makes the comment sound shallow and detracts from your point.

Also, I'm getting the feeling all the lols are meant to underline something clever and pointed, but I fail to see it. Can you clarify?

I think sometimes educators have to let people be motivated by some particular application to teach them something more important. Perhaps that is what the author is up to?

For the most part sets are used as a notational device, you almost never use any axioms about sets explicitly. If you asked most professional mathematicians to give you the ZFC axioms (or whatever) they would need to look them up.

That said, there is probably no reason for anyone to dig too deep into theory without understanding why it matters first. And I don't think it ever matters if you're just trying to solve a problem with ML. Even if you want to overlay a solution on an image of a Sudoku puzzle, you don't need to understand why a particular algorithm works in order to be able to use it. Theory only matters if you're either trying to formally prove something works or possibly to guide you down more productive research paths in developing new algorithms, though honestly I'm not even sure that's really true any more. Just gaining a ton of experience with various possible techniques might generate intuition that is as useful or even more useful than actually understanding anything.

I almost hate saying that, as I have a deep personal love of math theory and think it is worth studying to me at least for its own sake, but this is sort of in the vein of whoever invented the curveball almost certainly didn't know much about aerodynamics and didn't need to. Discovering something does work may happen decades or even centuries before anyone figures out why it works.

You develop your algorithm and you prove a bunch of theorems about it at the same time, incredibly difficult but totally wild!

There's some scattered projects like this https://github.com/OUPL/MLCert