This criticism seems to mostly be that he is associated with bad people who say naughty things. Maybe he is a bad scientist, but where are all the well designed scientific studies that show his conclusions are wrong? That's what I care about, but they don't exist because you would never get funding and even to propose running such a study would be career suicide.