If you are in a democratic country, your society has explicitly granted your government authority through your voting systems to be the ultimate arbiter and parts of it can and does get revoked as the will of the society changes.