It's the classic problem: you dislike it but it won't make you support or oppose.

For the ones who hate buildings: they will oppose anything.

In a world like that, every bit of ornamentation is a novel handle you give someone who will use it to pull you down, and they're going to try to extract as much surplus value as possible out of the construction process through various mechanisms.

So the simple thing is, you say things but do not care, so those who build do so by listening to those who care.