Exactly. Google has a nice internal solution where data at rest is encrypted with a record-unique (or bucket-unique) key and a separate system decides whether to give you that key based on whether you give it auth tokens that entitle you to access to that record. That way, having direct access to the datastore doesn't give you automatic access to everybody's data, and all access is auditable. (And data can be "deleted" wherever it has been replicated just by deleting that key for good at the centralized key store).

Obviously, some admins/sres still need to have full access to the key store, but that can be a very small group, as compared to a situation where "every Gmail engineer can read every user's email".

Edit: on reading the summary blurb from the "translucent databases" book link that @specialist posted, what I described above is very much along those lines.