I think the claim is not so much that you are or might be a crackpot, as that it's too easy to misdiagnose a genuine revolutionary as a crackpot. The canonical example for mathematicians is Ramanujan (https://en.wikipedia.org/wiki/Srinivasa_Ramanujan); Hardy would have been perfectly justified in thinking him a crackpot and throwing away his letters, and one of the most valuable collaborations in number theory would never have happened.