I don't believe an accusation is all that is needed. However, in a situation like this optics do matter. Employees need to trust that their superiors won't be part of the problem. He clearly didn't have that trust.
Also, standards are higher for executives.
But firing someone for allegations of something that happened at a previous employer, where there is no possible way the current employer's HR department can investigate it, seems entirely wrong. This means that someone can be wrongfully accused of something by some vengeful former employer/boss, and never get a job again because they'll be blacklisted. It is not unusual for employers to badmouth their former employees.
It is hard to see that this is generalizable to the circumstances of typical employees.