Or "Hey this guy is pretty good at business, lets give him low taxes to reward him and keep him happy."
I think its difficult for successful people to see how much they're lifted up by everyday people in society doing their best to incentivize meritocracy and creating an even playing field where the smart people can succeed.
Lets remember, the idea of the meritorious succeeding is rare in society. Up until recently you had to be a non-woman, non-slave, and at the very least come from a wealthy or political family. Or perhaps sidestep this by having powerful political patrons but then still be under the gun of, say, the inquisition.
Also when we look at many of the enlightenment thinkers and supporters, we see a slew of milquetoast intellects and ordinary people fighting against the old order of theocracies, monarchs, and dictators. Why did the smart need them to build the society they needed to succeed in? Newton wasn't writing about the freedom of man, instead he was obsessed over his place in the kingdom of heaven and went a bit mad about non-violent coin counterfeiters, advocating for their death, and often delivering it using the power of his position in the mint.
I think there's an Ayn Rand sized fallacy hole that you're falling into here.
In most historical human societies, the way to "succeed" was to threaten to kill other people unless they gave you what you wanted.
A couple of immediate objections come to mind. First, "intelligence" is difficult to define or measure. Things like standardized IQ tests tend to come with an implicit agenda. Second, thinking instead about "innovation", history surely shows that most innovation happens when humans group together (in farming communities, cities etc).
It's true though that in recent history, cities are more associated with social atomization. Overall it seems like a very fuzzy and complicated issue.
Sort of suggests that following the collective, being an ant in a colony may have negative effects on intelligence over a few generations. Natures law: "What you don't use you lose." You don't need higher intelligence because so many are chipping in to compensate.
Then again, balance is a fickle thing.