Black has a negative connotation because of its association with death. That is in fact the origin of the term "blacklist" - a list of people associated with the execution of Charles II's father. It's also the origin of the term "black day" - a day of death and, by extension, disaster.
While this association is not in any way "natural" or necessary, and it's not even universal, it is still extraordinarily old - dating all the way to Ancient Egypt and influencing European culture from there to now. And thus, it is extraordinarily hard to remove by playing language games with one word.
Why would you do that? Wearing black does not imply that a person is bad. "Blacklist/whitelist" implies that whatever is in the list is bad/good.