As long as you understand the risks of marginalizing people who live in the same society, it's "fine".

Just have to accept that once in a while one of them will do something bad and the majority will vote to further restrict your and their own freedoms in hopes of better security, not realizing it wasn't a security issue in the first place.