Where I disagree with you on "colour blindness" is that your statement assumes that gender and race are reasonable and obvious ways of categorizing people. How often do you group people by whether or not their earlobes are attached? Or by the curliness of their hair? We have made gender, race, etc important constructs. It would be difficult, but not impossible, to make them unimportant.
Accepting differences is a good thing. However the magnitude of the perceived difference can make accepting it either easier or harder
We are all different, but that doesn't mean we must be treated different at all times.
At work we are all workers. We may be treated differently depending on our position within the company, because that is relevant to our job and duties. But how is it relevant and useful to treat employees or colleagues differently because they are this or that? Does it bring any more profit, productivity or anything positive at all?
We can't be looking at these differences all the time, because most of the time they do not matter at all within the context. This is what people don't get.
EDIT: I'm definitely not saying that the subjective perceived differences in gender and race are real, or that bigotry and ignorance are cool.