This line of reasoning can be used to reject and remove access to literally any app or API. Why offer a contacts API if you're not allowed to use it? Who is trustworthy enough to have this privilege?

They should just remove all third party API's to access any user data. Then we can be certain of privacy, with only non-shady actors like Google, Samsung, Facebook, etc. accessing saved contacts.