any fitness pressure that would be making humans smarter
I don’t recall the study, it was either men in the US or Britton. But, income was positively correlated with number of children. Income and intelligence are also linked so that's evidence that right now intelligence is positively correlated with number of children. This may be strongly linked with prison time but again that links with intelligence.
The important thing to note is men often have children with more than one person and can have children vary late in life.
I do want to mention that larger social organizations were quite widespread in the Americas, despite the existence of hunter-gatherer tribes. The new world certainly wasn't an uncivilized wasteland when Europeans arrived.
Epigenetic factors can be triggered buy group size (domestication) as well. So those hunter-gatherer tribes might share the same genetic traits but express them differently based on their environment.