Might be kind of hard to "destigmatize" the underlying concept behind the various ways of saying someone isn't very smart.

The general tendency of most people to favor the in-group probably also provides enough of an underlying reality to make it hard to "destigmatize" the various ways of calling someone part of the out-group.