Don't forget war. I imagine when two groups of humans whether it be two caveman tribes or two Napoleonic armies, massive numbers of young males were killed off.