I think you've just got perceptual tuning for the human to human context. It's like how human faces that are 5% different can be very obvious to us, but it's much harder to tell (for example) two cats apart at the same difference level. All of your adaptive pressures are pulling in that direction.

But at a system-scale view of intelligence, humans are squashed together and it plays a pretty small role on outcomes. You should much prefer being rich than smart if your goal is 'success' by most metrics.

"I think you've just got perceptual tuning for the human to human context"

Not at all. I have to learn how to talk very differently to people on both ends of the IQ range. Wealth is a terrible proxy for IQ because it doesn't actually correlate very much due to accidents of birth and interests. Just look at Terrance Tao vs Trump's idiot sons.