Gorilla males are about 50% larger compared to human males being about 20% larger. There is much in our anatomy that implies humans behaved more as polyamorous tribes than solitary male harems.

If you're really interested, look up "Sex at Dawn"