"Ditto in the height of the Civil Rights era there was a massive market opportunity for restaurants that were willing to serve people of color - yet this largely didn't happen."

That's untrue. Black areas had high density of businesses that catered to blacks, and by large discrimination was legally mandated, not a market outcome. Business groups were in general in favor of dismantling segregation because it increased their potential market.