One thing I have seen done is taking filesystem (ex. ZFS) snapshots every n minutes on a secondary server. You can then restore the secondary and promote it to master.
Re:Update on the April 5th, 2017 Outage
by danappelxx on Friday April 07, 2017 @11:17AM
(#14060390)
Not an expert, but can't you just temporarily make the replica the master?
Not if a drop database was done - the replica would have followed and dropped it too
In this case, it looks like they could have, since the replica was time-delayed -- stop replication, then carefully let it continue replication up to the point you want, then switch it to master.
I wonder if there's anything to be done about reducing the duration of these outages. I suppose copying TBs of data around the world is simply always going to take hours.