It makes as much sense as taking an oboe player and asking them to conduct an orchestra, without any conducting experience.
Do you know how to handle investor relations and different factions on the board? Managing sales and marketing strategies? Assessing the regulatory environment in Europe and the legal risks of expansion in China? Can you speak confidently and charmingly at conferences with billionaires? Do you understand how to manage different interest rate environments? Is it obvious to you which law firms to hire for different needs?
This isn't just stuff you can learn on the fly. It takes many years of experience, and often a business degree in the middle, to become even moderately competent in this stuff.
To be charitable, for some of these, what's happening is that rather than leveraging their skills, they're leveraging their relationships. And that can be valuable, sometimes, but it's not a substitute for actually knowing what the hell you're doing.
The real problem is that far too many of these people—skills or no skills—primarily bring their egos to the table, and operate mostly based on that.
You "have yet to see evidence" because you haven't actually tried to learn accurate information about it.
There's only be limited studies done in this vain, and the results, while limited, seem to suggest that any reasonably smart person with knowledge of the industry could do a good job but the studies are pretty limited, and don't construct the circumstances as I described, so its hard to say its true or not true.