The demographics of start-up founders is generally biased toward young, affluent members of a community's ethnic majority. If you want to work for one of these companies, and "cultural fit" is high on the selection criteria, most start-ups will end up being staffed or run by the same sorts of people.

Working with diverse people that are interested in things that you're not is not predetermined to actually be a miserable experience. But it does require that you're open to integrating yourself with many other folks / cultures / etc.