Well it was taken down because it was representative of an atrocity, not because it was stolen.

Although, arguing down that route, war spoils are a tricky thing to discuss historically. "Stolen" is very different from "appropriated after successful war". Like I've said above, history and modern morals don't always mix. For that matter, current events and modern morals don't always mix. Take the Iraq oil situation, a bit more than a statue worth there.