You mean like the system in China? No thanks. Never underestimate the propensity of any government to violate human rights when given the power to do so.
If a choice has to be made between the two, a government that respects human rights is far preferable to a country without crime among citizens.
Have these people been convicted of crimes?
Because if they haven't, then yes, actually you are absolutely advocating restricting freedoms, and stumbling dangerously close to advocating redlining which has been illegal in the US for decades.
Edit:
Pardon my double post, I think a page got stuck loading on me somewhere.
A word of advice then: best state that immediately from the very beginning next time, instead of equivocating your way around the point for days. The minute you start talking about "curtailing freedom" there is one inextricable link that conjures in the mind from this type of language: the state.
Backing off it once the horse has been beaten into putty to say "oh I wasn't talking about the state" just looks like you're doing all of this for sport and not attempting to hold an actual meaningful position before finding a convenient escape hatch out of the discussion.
Have you never dealt with an HOA in your life? Have you never known someone who has?
I believe we need more defense in depth. We need our human rights to be protected by every level of government and informally by society itself.
It is a story as old as time.