There are some folks who are incredibly motivated and also fundamentally incapable. But for most people, the biggest "issue" is that there are multiple lucrative and lower-stress exit ramps.
Learning higher maths and physics is not intrinsically more difficult, but it is hard for most to maintain focus on a topic with only vague applications to anything and leads for most to financial ruin.
Software skills are more common merely by virtue of how applicable they are.
Given that the income delta between a researcher at CERN and a professional software engineer at Google is >10x I'd be skeptical that this selection always translates to intelligence. Anecdotally, many PhD Physicists transfer into software for monetary reasons and do not come across as intrinsically smarter than the average engineer of equivalent experience.
The median person who can drive, pay taxes, be a loving partner, and parent is just ... disappointingly not smart if you've been surrounded by brainiacs for most of your life. Some very bright people have other personality quirks that make it hard for them to integrate new ideas outside their comfort zone.
We need better people.