Perhaps because they live and impact the society you live in, from partaking in government, to voting, amongst other things which may potentially have an impact on your life (such as preventing you from having a sane discussion). I shouldn't even have to mention the Nazis of the 1930s as an extreme example to drive the point home on how anti-semite nut jobs could result in bad things.