rsnapshot (http://rsnapshot.org/) against all of our openbsd servers over the internet (via ssh) to an off-site machine. runs every few hours and keeps 6 hourly, 7 daily, 4 weekly, and 3 monthly backups. once a server's initial full rsync is done, the incrementals finish very quickly even on servers with lots of changing data (mail server messages, web server logs, etc.)

regular mysql dumps are taken from all databases on all servers every so often in case the rsync'd mysql binary files won't restore.