The "why bother" effect became very strong in me after my last long-term relationship failed, and I noticed that my advancing age made it even more difficult to get anyone interested in starting a new thing.
My personal disillusionment reached its peak when I realized that dating and marriage are one of these treadmills that just eats away at your life. People who are physically attractive, well-educated, and wealthy are in my opinion much less likely to feel this drain - both because they have more attractive options to choose from and because they tend to have a more gratifying life in general. Whereas people like me would have to expend a considerable amount of energy just to even get an opportunity, only to end up with a less-than-ideal match (because if I'm "settling" for someone, odds are she's also somehow forced to "settle" with a less-than-ideal partner such as me), and I'd have to spend an inordinate amount of effort just to keep up in the long run. If you're someone who is constantly getting the feedback that you're inadequate, every step of this is degrading. So why do it?
So the realization is that striving for partnership is not worth it for many men (and I suspect for some women, too). It's perfectly possible to lead a fulfilling life without it, once you stop running after the proverbial carrot that society would like to dangle in front of you. Instead of being someone whose primary attribute is being a sub-par partner, I can concentrate on being actually valuable at other things that matter to me.
In general, even happy "normal" relationships are a ton of work. They also anchor you to certain expectations and when you try to change something in life, you risk upsetting your partner. Want to study for an exam? Good luck having scandals about "not enough attention."
Furthermore, a lot of the time your partner won't grow with you. That's why you see a lot of people divorcing once they "make it".
For all the stories about large alimony, such situation not the norm.
So I would expect divorce issues not to influence causual sex rates, but rather marriage rates.
And, honestly, it doesn't seem like women are terribly upset about this. This seems to be the world (almost) everybody wants.
1. Video games give us fulfillment. If the reason you're looking for intimate relationships is fulfillment in life and video games partly fill that hole, then you're less likely to look for intimate relationships. Men tend to make up a much greater share of games that require a large time investment ("hardcore games").
2. Social media gives us some human contact, but it rarely leads to intimate contact. It scratches the itch of human interaction without needing us to leave the room.
3. Porn. The easy access to porn means that there's less of a reason to search for intimate contact. Men tend to be visual. When you're online, then images of a beautiful woman are only a few clicks away. It doesn't come with any of the baggage relationships come with either.
3. Dating apps. Women on dating apps seem to be harsher when judging the attractiveness of men than men are. This leads to a situation where average looking men will find it harder to find intimate partners on these services. They can't really make up for it by being smart or witty either.
4. Relationships are difficult to start. If a man approaches a woman and fails, then (sometimes) he's considered creepy. In many cases it's simply a misreading of expectations. For example, some women think that a man should always approach first, but some women think that unless a woman explicitly shows interest then a man approaching her is creepy. It's risky for a man to start the interaction, so they are more apprehensive. At least this is the impression that I got based on what I've seen online, particularly Twitter.
5. Divorce. In many of the stories you hear about separation and divorce, it's the man that gets shafted. Women are more likely to get the kids and, at least in the stories, they might get a large alimony too. This means that the man would be severely hampered in any future relationships they seek, because some of their income goes to the former spouse.
Some men add these together and then ask themselves "why bother?" Why risk approaching a woman, if instead you could be playing video games? They are as much fun, and hey, chances are that she wouldn't be into you anyway. As a result, some men opt out from this part of society.
I'm sure there are many other small issues like this that I couldn't think of, but these are certainly things that made me question whether relationships are even worth it. Every time you hear about things like "toxic masculinity" or some such it ever so slightly chips away at some men's interest in participating in society. The small chips have added up. Couple that with compelling alternatives on how to spend your time and I think you get the result we have.
You could also argue that we're simply following in the footsteps of Japan.