I encourage you to look up the history of segregation in the U.S. That approach works out very badly.
but I see your point and concede that it can be a dangerous slippery slope.
Very different, but still a long way to go. I remember reading that in ~1959 (this is from memory, so all details are a bit hazy) 95% of Americans opposed legalizing interracial marriage; now of course a similar number support its legality - it's a non-issue. Look at the boom in the black middle class (it's amazing what people do when you give them a chance). Look at LGBTQ rights and the incredible advance of women in society.
On the other hand, look at who won the November election, offering the most brazen racially and religiously prejudiced, hateful messages of any major party candidate, possibly ever, and openly partnering with the most powerful agent of white supremacy in the U.S. (Bannon). Look at how minorities are treated by law enforcement in many places. Look at the overwhelming prevalence of white-skinned people (and men) in movies, IT (name the top 10 industry leaders in the last 25 years), national and state government, Fortune 500 CEOs, etc etc.
It is not fair one way or the other. You have also put yourself in the unenviable position of determining what can and can not be controlled by an individual - this will not end well.
| ...equal enjoyment of the goods, services, facilities, and privileges, advantages, and accommodations of any place of public accommodation, as defined in this section, without discrimination or segregation on the ground of race, color, religion, or national origin.
IE, a specific group of people in specific set of places. That does not include discrimination against "things people can't control" in any and all establishments.