You're making a good point. We can, however, write software that makes certain guarantees on the client side. If I'm the only one with the encryption keys, does it really matter what the server does?

This is the basic idea behind zero-knowledge systems using homomorphic encryption to do very cool things.