This kind of morality is between you and your Rabbi. We are allowed to look down on it all we like. It has no relevance to law or keeping your job because if you allow this kind of thing to have relevance it never ends. Rank selfishness comes out in so many ways.
"You got a parking fine, your social conscience is clearly lacking..."
Hard to estimate it precisely but less than 1% is likely to be accurate, no?
Even reading the history of European nobility, it is amazing how often you'd come across married nobles who didn't even live in the same city as their partner, and who openly had mistresses or young handsome men hanging around all the time.
I've also talked to Chinese business men who lamented that they are expected to have an overseas girlfriend, almost as a sign of status.
And hell, Saturday Night Live has been joking about rich old ladies and their love of hiring pool boys for literally decades now.
My point is not to normalize cheating, but rather that all sorts of arrangements have existed throughout time and different cultures and instantly saying "he's sleeping with someone who isn't his wife!' doesn't mean that person is devoid of morals.