Depends how obvious it is .

Put a car repair sign up then yeah. Be more “quiet” about it and claim and you are fixing a friends car and it becomes less easy / an ordeal for the city or county to act. Unfortunately the car repair guy was not honest and that was an issue.

Party house was a bit of an ordeal.

HOAs tend to fill a lot of zoning gaps.