The affair isn't the issue, or at least not the only issue, it's who he had the affair with.

At the end of the day Jensen chose to have an affair with a senior officer of a company strongly suspected of bypassing trade restrictions and should have known that it would be a terrible look for Nvidia and himself. Even if he didn't suspect her of being an agent of China, even if he didn't believe the allegations against Mega speed, even if he had no intentions of passing confidential information or aiding in bypassing trade restrictions. He knew it would carry the appearance of all those things.

That is the unforgivable lapse in judgement.