The attire of actual drug dealers is irrelevant, what matters is the imaginary drug dealer dress code in the minds of the public. One of the more subtle forms of privilege is that some groups have more "freedom of fashion" than others. Richard Branson can do "a very expensive version of whatever the Dude wears in The Big Lebowski", because there are cultural stereotypes available to him to play with that are unavailable to other skin tones. Likewise, a white person wearing impeccable business in the wrong environment will evoke the worst of wallstreet, whereas a black person wearing the same would be comfortably grounded in a proud tradition of dressing upward.

depends on what kinds of drugs they are selling in my experience, color of their skin is irrelevant for the most part

I'd suspect it depends far more on how their clientelle dress. And that the dealer aims for just a shade higher. As do most salespeople.