Of course this is not a new idea. Think Wagner, for example, a notorious antisemite. Or consider a utilitarian view. It is to me somewhat alarming, though, the swiftness with which we pass judgment these days, and I think it's worth considering the impact.
When looking at historical figures context is everything. I judge men who sexually harassed women in the 80s way less harshly than men who harass women today, why? Because in the 80s society still told you, 'this is ok, this is normal, feel free' for many forms of sexual harassment. That doesn't exempt them from culpability but given the degree of effort that would have been necessary at the time to understand what was wrong with their actions we should temper our condemnation of them.
The point this leads to is: If Feynman was alive today, sexually harassing women, that's significantly worse than similar behavior in his heyday; making your question hard to give a meaningful answer to.
Basically we're a bunch of crybabies today.
In this more enlightened age, people should call the harrasser out on their behavior before it becomes a big problem, and the harrasser should have enough awareness to stop the damaging behavior, which is not necessary for their work.
Sir Tim Hunt might be a better comparison. One (possibly) out-of-bounds joke and he's kicked out of science, no matter how bad the results for humanity. Not to mention his students, some of whom are women.
See these three letters before his name? The ones that spell out S-I-R? Those three letters, along with other titles and positions he held, come with some very tangible benefits that none of use have -- and also some very tangible responsibilities, which, again, aren't bestowed upon everyone else.
He wasn't forced to be Sir Tim Hunt, the VIP, the public speaker at conferences. He accepted the titles, the positions, and the responsibilities that came with them (unlike, say, Grigori Perelman[1], who, by showing the middle finger to the establishment, has also freed himself from any burdens of the sort - or Alexander Grothendieck[2]).
One of the responsibilities of being Sir Tim Hunt is that you can't be a self-titled chauvinistic pig that cracks stupid sexist jokes.
There is an unspoken contract that Sir Tim Hunt violated, and suffered for doing so.
There is absolutely nothing new here. The rules of the society have not changed. Jim Simons[3] was kicked out of his high position in a crypto lab in the 70's for criticizing the Vietnam war in an interview to a small paper (a well-known story that hasn't made it to Wikipedia yet). One (possibly) out-of-bounds remark.
I am only giving examples of very-high-profile mathematicians here, but they are quite representative.
[1] https://en.wikipedia.org/wiki/Grigori_Perelman
[2] https://en.wikipedia.org/wiki/Alexander_Grothendieck
[3]
He literally called himself a chauvinist monster in that joke. Hence self-titled.
My point is that people in his position are held to stricter standards. The standards include carefully choosing what to say.
Sure, he'd make a great babysitter for your kids - he just didn't make a great public person.