Sure, sucks. But anyway, would you like to live in a place where the majority of people don't like you? It's going to be a fact with our without referendums or laws.

If you get a different result under representative democracy, the minority must be assumed to have more actual (real world) power than mere headcount indicates.

In that case, direct democracy is indeed broken, but the reason is not "discrimination" against those who are actually pretty powerful, but eventual instability from the system's inability to act as a predictive proxy for more direct (less civilized) resolution.