As I just amended my comment up-thread to say: allergies. Food allergies can kill you, and nut oils (nuts being a fairly common, and often severe allergen) are sometimes used used as an olive oil adulterant.
Adulterated olive oil isn't even labeled that it's adulterated, let alone what with.
Because you miss out on the chance to get angry?
As other have mentioned the inferior oils might not be as healthy, this might be a reason to care not just 'cause'.