Fathers have multiple physical adaptations for fatherhood that are very important. The idea that fathers experience no hormonal or physical changes is not scientific.
Much like mothers, though, if they remove themselves from the proximity of their children these effects cease.
In summary, men are more likely than women to be reproductively successful with a low effort child rearing strategy. Is it the best strategy overall in modern society? I don't think so, but you can see how we'd evolve that way.
1. Neither the premise of spreading genes with "little effort" nor the bit about incentivization is actually true unless you have a very odd definition "less time". While lifelong monogamy may not be an obvious strategy, communal child rearing that involves fathers has been the norm for most of recorded history in all civilizations, with poor and absent fathering nearly always being villified unless needs of the community outweighed it.
2. Lots of women have multiple children, no problem. Many elect to go back through it again. This idea that pregnancy wracks every woman is just ridiculous.
It's not even clear we evolved that way. Looking at our near genetic relatives, they don't actually do things the way you're describing and communal living isn't "a group of females desperately trying to avoid pregnancy as males raid and ravage them."
That said, I agree it's a shame when it actually plays out that way and our society as a whole (and even fathers in modern society) lose out.