I think that was explained quite well in the article, and while the symbol has had a long history I think it is fair to say that these days the majority of people would see the symbol and think "Nazi".

Removing it feels like a reasonable thing to do, though I'd probably draw the line at chisel it off old-buildings, etc.