I'm not sure I understand your reasoning - why would some negative effects of an intervention automatically invalidate the positive effects? If you don't think there are plenty of positive effects, feel free to speak to somebody from the German Jewish community and see how they think about it.

Which is my point - in a democracy restrictions to the rights of individuals are there to protect the rights of others. This rationale is shared by many restrictions to individual rights, hence the speed limit example.