You don't want to give unlimited access, but once someone authorizes an individual as a contact, the contact should be kept up to date automatically if a phone number or email address changes.

(Also, partner organizations might want to share directories easily; I can see giving a full access to your directory to a contracting/temp agency you use, rather than giving them accounts. I'm sure there are better examples.)