Your mistake is you're only thinking about it from the perspective of the stereotyper. Now think about the negative impact (career, legal system, etc) on the person being incorrectly stereotyped.