It's not about rights, which there is no scarcity of, it's about the fact that there's always some social competition for scarce resources (college admissions, jobs, dating opportunities, grants, cultural conversations, etc) and if one group currently has a significant chunk of those they will feel threatened that their share of those resources is being taken away.