If I may suggest a slight rephrase: If you are concerned about RELIABLY destroying your data (which involves being able to verify for yourself that it was thoroughly destroyed when you think it was destroyed), you should keep that data on your own devices.

If your data is stored on a server that someone else controls, you can't reliably destroy that data. Even if it appears to have been destroyed immediately, there could be other copies somewhere. So wherever possible, sensitive data should not be stored on someone else's server. Even in encrypted form.

Or run on an encrypted volume that you scrub prior to killing off the instance.

you shouldn't rely on your hosting provider to secure/shred your data, do it yourself.

encfs is really easy to setup and use, everybody should be using it, especially on VPS's:

    $ sudo apt-get install encfs

or:

    $ sudo yum -y install fuse-encfs

then:

    $ sudo encfs ~/.encrypted /home/private

the /home/private folder is where you place your files (web sites, etc.). ~/.encrypted is where the encrypted version is stored. when prompted hit 'p' for the default paranoid mode, enter a password and you're done. read more about it on the homepage[0]

When you are done with a server and need to secure delete, use shred -- which is installed on both RHEL and Debian based distributions:

    $ sudo shred -fz ~/.encrypted 
    $ sudo shred -fz /home/private

[0] http://www.arg0.net/encfs

https://defuse.ca/audits/encfs.htm https://defuse.ca/audits/ecryptfs.htm

Probably better to use ecryptfs or dm-crypt.

encfs is not completely transparent, so don't just blindly do this, you need to check that your data/workflow does not interact badly.

Two examples: (Depending on the settings you choose) hard links might not work at all, and renaming can be very very slow. Both of these are common in certain applications.

Wouldn't an encrypted volume be essentially random data to anyone who doesn't have the key? So all you'd need to do is scrub any traces of the key itself.

Sure, but if't it's sensitive data, the extra couple minutes is probably worthwhile.