Not quite. We are descended from sexual pairs whose children survived long enough to reproduce. Often, it took at least both mother and father to keep children alive. And human female reproduction selects for males who stick around, primarily by hiding most signs of ovulation, unlike many other mammals with obvious signs of momentary fertility.
But you're not wrong either. Both male and female reproductive strategies want to maximize number and viability of offspring, but the female cost is much higher than the male cost in time and energy, so males can theoretically roam more easily. In the past, mobility and low populations generally limited that ability (though Genghis Khan managed). That's the disconnect between prehistoric and modern humans that porn and dating apps exploit.
> like all men, I do have as much sex as I possibly can
I used to think fat people were fat simply by choice. I mean, they are the only ones lifting the fork, right? But I no longer think it's this simple. We are the result of billions of years of evolution in a world where food was scarce. Those who didn't compete for food perished. Those who ate as much as they could when it was available survived. Nobody got fat because it was simply not possible.
Sex is similar. We are descended from males who mated with as many women as they could. Maybe not all men are like me, but there is a part of my brain that literally wants to mate with every attractive woman I see. Obviously this is not possible for many reasons, but like all men, I do have as much sex as I possibly can, it's just not that much (not more than once a day). But you can read stories from rock stars and the like who do have access to much more sex and it can ruin their lives. This high availability is analogous to the high availability of food in modern times.
Porn provides this high availability for normal people. It's like constantly available junk food. Even if I don't go looking for it, I am surrounded by it everywhere I look. Every other Youtube video is a woman with cleavage in the thumbnail. It's like being a fat person constantly surrounded with a buffet wherever I go. And remember, it's porn. It's not like I'm surrounded by cabbages and carrots, I'm surrounded by cakes and biscuits.
I fail to see how climbing a mountain or starting a business or any of the other wholesome pursuits you might be thinking of would curb the desire to eat or mate. The underlying problem here is deeply rooted in our animal brains. The best way to curb the eating problem is to ensure food isn't available. Don't keep any ready-to-eat food in your house. The best way to curb the sex problem is exactly the same.