In my experience, most antisemites are fairly stupid* and it is hard for them to understand the logic which says "just because a small subset of a certain ethnic/cultural group has disproportionate influence over the society in which they live does not mean that every member of that ethnic/cultural group has disproportionate influence over that society".
This fallacy is sort of like thinking that, because most NBA players are African-American, it means that most or all African-Americans must be really good at basketball. Rather obviously flawed logic.
*Certainly not all, though - the more intelligent ones probably contribute the bulk of antisemitic literature.
Btw, accusing someone of something should or should not be a bannable offence independently from the accused ethnicity or religion. You don't want to have classes of people who can or cannot be accused of something on a "racial" basis, do you?