Are you american? The term isn't used to describe skin color per se, but to classify someone as a member of a certain group with attendant right/privileges and cultural traits: https://en.wikipedia.org/wiki/One-drop_rule

Recently 'blacks' weren't allowed in certain venues/institutions and were required to take a generally subservient stance vis a vis 'whites'. White also receive(d) preference for admittance to the country and citizenship.

The non-denotative nature of the language is well-discussed: http://forward.com/opinion/318667/im-a-mizrahi-jew-do-i-coun...