The truth is probably somewhere in between though. The time/energy constraint is what's most apparent to me since working full time.
As an adult you have to cram cooking, cleaning, hobbies, relationship building, etc all into your small limited evening windows and weekends.
If you want to be good at something you need to put time and energy into it. This means that you can't be good at everything.
So if you like to spend your time reading, maybe selecting nicer clothes (that often need ironing or handwashing, i.e. time) will maybe fall by the wayside, because you prefer reading books. Meaning that you look worse than you could, because you prefer your hobby.
Similarly somebody who spends a lot of time on their outward appearance might not have the time left to be as interesting as they could be.
In the end of course you may have talent or natural good looks. But as soon as you start needing to think about where to allocate time you will face tradeoffs.
These people have resumés that look like they rolled a 20 for every one of their stats.
"America's Finest".
Same with the dating situation. Must be more comforting to think "I'm not the most attractive person ever, but at least I'm sane and many of the more attractive people aren't" rather than "There are probably people out there who are both more attractive and with a nicer personality".
I also suspect people tend to think everything in life must have tradeoffs, D&D character points style. Like, you have X points to go around in total and if a percentage of that goes to intelligence/skill, there must be very little left over to put into charisma.
You can see that a lot in films and TV shows, where being popular or attractive is often treated as if it's incompatible with being intelligent or a nice person. But that's not reality, and some intelligent/attractive people are also perfectly nice, sane people.