Because the government can. Ask the Jews in Nazi Germany how "rights" worked for them. Prior to Hitler coming to power, Germany was the most advanced and liberalized society of its time.