Because it easily becomes "let's prohibit any small minority group the people in power find distasteful or unpopular." The net hates/hated Scientologists, for example, but I'm sure you could find others. Jehovah's Witnesses generally are disliked or have only negative public perception.