You’re not “harmed” by an ugly building. Annoyed, maybe, but you’re not harmed. Ugly is subjective. Even if the majority agree that it is ugly, there’s likely a group that enjoys it. Should they be denied the ability to have architecture that pleases them?