So I'm used to one of the things listed in this article. I'm fine with this preconception people have, as it's not that much of a problem on it's own, and it's easy to take it in my stride. However, it's the gestalt of all those things that the author is talking about that is the problem. They all work together, and if I experienced all of those working against me, I'd feel unfairly treated as well. "People fear me unfairly on the street? No worries, I'll be at work soon, and I don't have to deal with that shit there..."
However, I strongly object to the idea that it's okay to discriminate based on unalterable characteristics as long as you're not discriminating against a member of a legally protected group.
I never demanded you stop crossing the road because you're scared of specific groups, and for that matter neither did the author of the article, although our situations are obviously very different. It has merely been pointed out that doing certain legal things can have a negative impact on other people. In my case, I am pointing out that crossing the street to avoid someone has a negative impact on whoever is being avoided, regardless of race. This is clearly not the same as calling for legislation to prevent unlawful street crossing.
As you have noted, the government won't stop anyone from calling you a dick. If you don't like being called a dick, well, that's too bad.
So they're doing the right thing.
The notion of justifiable discrimination will take you to a very scary place if you apply it consistently.