Technically it is NOT google's fault to open those access to Apps.
Just like credit card CVV, the right to grant you the access to it, prohibit you to store in your server. For personal information, I think social networks need to be held responsible to live up to the same standard.
By granting Facebook permission to read my contact, my understanding is that they should only use my contact to match against their DB and find those who are on FB. I don't think this require them to persist all my contact/conversation history in their own server.
That's why, of course, chips for physical purchase have moved to one-time codes, effectively, so that your credit card number can't be stored without permission. Ideally, someday our online purchases will work the same way.