One additional thing you can do is add a "command=..." parameter to your ssh authorized_keys file to limit what can be done with that key. For example, you can set it to run a script which only allows new files to be added to the backup, but not deleted.
So for the last 6 years or so (we've been providing offsite backup since 2001) we have offered "pull backups" to our customers that request it. We give them our public SSH key, and we log in and rsync their data back to us.
Also, RE: the previous comment about not having your data consolidated to a single provider, we run an ad on reddit regularly making the same point:
http://www.reddit.com/comments/hg9oa/your_platform_is_on_aws...