Having been involved in a large number of interviews as an interviewer, my experience tells me the issue is that many highly-competent engineers are incompetent interviewers.
To illustrate my point, here is an imaginary example in the extreme scenario. I once spent a large amount of spare time in graph theory. So I could easily fail at least 95% of candidates by giving algo questions in graph theory. The questions are easy to describe, so they are perfect for interviews in this sense. But I would have done my team a big disservice by turning away potentially very good candidates, and I would have harmed our company's reputation.
Edit: typos and grammers
I have never understood what application Mensa-quiz puzzles and algorithms trivia questions have to the day-to-day work of making things go. It seems like a smart-person pissing contest. Maybe that that's what the bulk of their work time consists of? I hope not, but I wouldn't be entirely shocked.
A previous employer loved them some graph problems for interviews. I got in an argument with one of the senior engineers, and demanded I be shown where in our codebase we used graph algorithms. The answer: nowhere. But you had to know your way around them to get hired there.
Now when I interview, I refuse to study them. I think this serves me well in helping me avoid jobs I wouldn't enjoy anyway.
An interviewer once essentially demanded I invent morris traversal on the whiteboard in front of him. I'm not sure how that related to my ability to be an ml engineer.
In fact, a long time ago an accountant used this very technique to decide that I was a smart software guy. He then hired me even though he knew nothing about computers and software.
If an engineer can contribute in a pair programming session then you know they have something to contribute in your development team later, even if you don't pair all the time.
when i run in to an interview like this, it tells me i dont want to work with these people.
thank you for this golden nugget. its like thinking you can build the best basketball team by getting the most athletic looking players, except much worse. i guess the conventional wisdom is that good software engineers are like mathematicians, but in my experience they are more like craftsmen, like a cabinet maker or ship builder. i suppose its contextual to the software problem, but analytical, deductive aptitude, while helpful, seems to pale in comparison to what i consider the craft (consistent style, naming, testing, pacing, working together and communicating, effective editor and shell use, knowing how the fuck to use git, etc) which have very little to do with analytical, deductive aptitude.
This industry's biggest hiring/talent problem isn't "posuers." The biggest problem is the persistent and incorrect belief that applied mathematicians (CS majors) can easily learn the skils needed to engineer complex systems as they go (or, worse, that said skills are trivial or even irrelevant compared to the ability to recite textbook DS/Algorithms trivia) and that they are also able to adequately determine the capabilities of other similarly trained persons to do the same mainly by relying on textbook test problems (albeit sometimes with a twist).
In other words the (undisputed, in my opinion) existence of people who can't write a simple for-loop doesn't justify the absurdities seen in modern software interviews.