News for hackers. Stuff that matters.
Re:AWS Data Pipeline by mseebach on Friday December 21, 2012 @10:48AM (#4953088)
It might not be appropriate for you, but a good way to handle MySQL backups is to maintain a mirror. This has the added benefit of being available as a fail-over and as a secondary instance where you can run reports or test long-running queries on current data without the risk of taking prod down.
Re:AWS Data Pipeline by jacques_chester on Friday December 21, 2012 @11:30AM (#4953315)
> It might not be appropriate for you, but a good way to handle MySQL backups is to maintain a mirror.

    DELETE * FROM business_critical_data; WHERE obsolete = true;
You were saying? :D
Re:AWS Data Pipeline by iamjustlooking on Friday December 21, 2012 @11:45AM (#4953394)
You can run your daily/hourly backups on the mirror and not impact performance on your main database.
Re:AWS Data Pipeline by jacques_chester on Friday December 21, 2012 @08:51PM (#4955452)
Right. But the grandparent comment was suggestive of the possibility that he or she wanted the mirror to fulfil multiple roles, including being the backup.
Re:AWS Data Pipeline by Negitivefrags on Friday December 21, 2012 @02:11PM (#4954028)
The mirror is to run the backups on, not the backup itself.
Re:AWS Data Pipeline by dsl on Friday December 21, 2012 @03:03PM (#4954270)
CHANGE MASTER TO MASTER_DELAY = 3600;

Your slave will always be 5 minutes behind the master. You were saying?

Re:AWS Data Pipeline by jacques_chester on Friday December 21, 2012 @09:06PM (#4955491)
If this is your tactic, I'd think the binary log would be the winning way to rollback a bad delete.
Re:AWS Data Pipeline by mseebach on Friday December 21, 2012 @11:45AM (#4953389)
Well, with a functional mirror to run manual queries against, you wouldn't be running the risk of running such a query on your prod DB anyway. But yeah, you still need a dump.
Re:AWS Data Pipeline by jacques_chester on Friday December 21, 2012 @09:29PM (#4955545)
There's lots of disaster reports that start with "shouldn't" and "normally we wouldn't" :)

http://www.taobackup.com/

Re:AWS Data Pipeline by mcos on Friday December 21, 2012 @11:10AM (#4953194)
Thanks, I've already tried that. My main issue is EBS performance when writing the dump file to disk. The backups themselves don't impact on database performace much, but writing up to 20 Gigs of a dump file to an EBS disk on a nightly basis is extremely slow. Maybe this Data Pipeline service will help bypass that.
Re:AWS Data Pipeline by 1SaltwaterC on Friday December 21, 2012 @02:23PM (#4954072)
Have you tried piping the dump directly to a compression tool? We use pbzip2 for our dumps which works great if you have some CPU power to spare. The largest was around 8 Gigs uncompressed, but the total size of the plain text dumps is over 20 Gigs. Hardly an issue for EBS that way. Did kill the DB for a few minutes several times before using cstream to cap the dump bw.
Re:AWS Data Pipeline by jeffbarr on Friday December 21, 2012 @12:00PM (#4953457)
Have you set up Provisioned IOPS for the volume?