If the person in question is not really unpleasant (and I don't know that they are not, but let's take it as given for a moment), my suggestion would be doing some strong networking. If you interact with people at conferences and are clearly not a bad person, then it's easier to defend yourself against the accusations by pointing out their variance with actual experience. In a perfect world, yeah, that work should not be necessary, but nobody's accused the world of being perfect in a long time.
(Ironically, if you pull this off and manage to build a new reputation on your own, then you might actually be able to safely sue.)
We're only hearing one side, but given his professional accomplishment of people using his textbook in Masters courses, this comes across as a personal vendetta. You don't say those sort of things about someone who clearly does have talent unless you want to cause harm.
If you win enough in damages then you don't need to worry too much about earning in the future.
No idea how likely that is, particularly after paying attorneys' fees.
I've met plenty of stupid people who can pass exams.
If "stupid people" are passing these exams, maybe these are not good exams either.
Finally, thinking in terms of "stupid people" also incorrectly frames intelligence as a scalar, is unhelpful and lacks descriptive power of how one's mental capacity falls short: Are they socially clueless? Are they forgetful? Are they not able to put in the mental work when it comes to answering questions?
Also against the universities for breach of the blacklisting prohibitions in the employment act. Their lack of checks and balances against this informal blacklist has left them wide open to be sued.