Why would an AI researcher need to be an expert programmer?

Furthermore, why would an AI researcher need to have intricate understandings of the inner workings of modern CPUs?

Lex is not a software engineer. Lex is not an electrical engineer.

I know people actively working in the AI/ML field who don't even have a B.A./B.S. - they're literally self-taught from watching YouTube videos; reading books, textbooks, articles; and listening to podcasts. And after all, wasn't that supposed to be the ultimate end-goal of all this high technology we've created for ourselves? To smash the Ivory Tower? To democratize knowledge for anyone who's willing to pursue it? These people don't understand the finer points of programming and they damn sure don't understand the finer points of CPU design... why should they? You have finite time in your day, and every hour you spend understanding the intricacies of one thing, that's an hour you've lost to understanding something else.

The promise of the Internet was supposed to be something to interconnect humanity so we could share amongst each other. Well part of that promise is relying on the expertise of others to help us, instead of having to acquire that expertise for ourselves.

Branch prediction as a concept is not a finer point of a CPU design - sure it's up to you to learn about it or not, but understanding the basics of writing CPU friendly code is really not hard in the slightest. AI workloads are some of the heaviest around at the moment, so a little can go a long way.

I am entirely self-taught in CS and I know what I know, I'm not inflicting it on anyone else I'm just saying Jim Keller doesn't do these very often and I want to hear him in more depth.

Research, and academia in particular, are very status/authority oriented, for better or worse. People with no credentials succeeding is not unheard of, but is extremely rare. Especially these days, when you’ll be competing with hordes of PhDs being pumped out by universities every year.

I found it the other way around, I got enlightened about many subjects and you took a peak on how Jim Keller perceives some problems and how he breaks them down - really remarkable.

For example, I really liked these two approaches regarding Moore's Law (Jim Keller: https://www.youtube.com/watch?v=c01BlUDIlK4 , and David Patterson : https://www.youtube.com/watch?v=c01BlUDIlK4 )

I liked those parts, but they were strategy and I really want to hear him talk tactics too. Details on the tools and techniques they actually use to design processors are really hard to come by.

He does know what branch prediction is, he is a CS graduate, That was just for his audience.

If it was just for his audience why does he say "that blows my mind" or something to that effect?

To make the show more engaging? “Oh, yeah, everybody knows that, whatever” is a quick way to turn your less knowledgeable audience off.