News for hackers. Stuff that matters.
Re:Let's Stop Talking About "Backups" by jfoutz on Monday December 14, 2009 @12:05PM (#994471)
backups are for suckers. keep the data on a few different spinning disks. if you can solve data synch between two sites, just keep your data synched.

it's much better to ask yourself how long to replicate your existing system then how to back up. pxe boot to a kernel that you can install over the network with, bcfg2 to get the thing up to spec, start copying data.

a lot of machines can be back and configured in 5 minutes.

that said, i'm not you. i don't have terabytes of data to do statistics on. maybe there are other horrible details i'm forgetting. fast rebuilding is a pretty awesome strategy for a lot of cases.

Re:Let's Stop Talking About "Backups" by mechanical_fish on Monday December 14, 2009 @12:14PM (#994505)
maybe there are other horrible details i'm forgetting

Yes.

Why should I bother to write this? I'll outsource the task to the authors of High Performance MySQL, Second Edition, page 475:

Backup Myth #1: "I Use Replication As a Backup"

This is a mistake we see quite often. A replication slave is not a backup. Neither is a RAID array. To see why, consider this: will they help you get back all your data if you accidentally execute DROP DATABASE on your production database? RAID and replication don't pass even this simple test. Not only are they not backups, they're not a substitute for backups. Nothing but backups fill the need for backups.

Re:Let's Stop Talking About "Backups" by diego on Monday December 14, 2009 @01:02PM (#994626)
This is the incremental backup script I use on my Linux box at home, a quick-and-dirty imitation of what Time Machine does. Obviously $HOME/backup is a different physical disk. Feel free to improve on this.

--------

#!/bin/bash

HOME=

date=`date "+%Y-%m-%dT%H:%M:%S"`

rsync -aP --link-dest=$HOME/backup/current /home

$HOME/backup/back-$date

rsync -aP --link-dest=$HOME/backup/current /etc $HOME/backup/back-$date

rm $HOME/backup/current

ln -s back-$date $HOME/backup/current

#see if the disk is getting full

FREE=`df -lk|grep sdb1|awk -F" " '{print $5}'|awk -F"%" '{print $1}'`

#alert me if the backup disk is getting full.

T=80

if [ "$FREE" -gt "$T" ]

then

    df -lk| mail $myaddress -s"disk alert $T% capacity"
else

    echo "backup disk is less than $T full"
fi
Re:Let's Stop Talking About "Backups" by Locke1689 on Monday December 14, 2009 @03:45PM (#995045)
You should think about keeping an offsite backup if possible. A second disk won't necessarily help if your computer gets dropped in a pool, house catches on fire, etc.
Re:Let's Stop Talking About "Backups" by jfoutz on Monday December 14, 2009 @01:01PM (#994625)
Completely reasonable response, forgive me for being an inarticulate noob in my post.

I thought delayed replication was one of the main strategies they advocated in that book. I don't have it on hand. my mistake.

Re:Let's Stop Talking About "Backups" by sandGorgon on Tuesday December 15, 2009 @02:31AM (#996112)
that is very interesting - I though replication was the nearest to realtime "backup" as one could get. Is there any other way, one can do incremental, realtime backups for, say, mysql/postgres ?
Re:Let's Stop Talking About "Backups" by gvb on Monday December 14, 2009 @12:23PM (#994527)
Obligatory xkcd link: "Bobby Tables" http://xkcd.com/327/
Re:Let's Stop Talking About "Backups" by idlewords on Monday December 14, 2009 @12:14PM (#994507)
Replication doesn't defend you against deleting the wrong file, or messing up an update to your database, or any of a large class of PEBKAC issues.

It's also not useful if your main files get corrupted and you diligently propagate the corruption. See: ma.gnolia

Re:Let's Stop Talking About "Backups" by jfoutz on Monday December 14, 2009 @01:04PM (#994632)
Setting aside the database, file problems are fairly easily solved via svn, .snapshot or a good configuration engine.

As far as the database goes, I'm big on stored procs + archive tables, but i'll leave that to the grown ups ;)

Re:Let's Stop Talking About "Backups" by masomenos on Monday December 14, 2009 @03:16PM (#994980)
...until you forget to check in important changes, or neglect to add a seemingly trivial configuration file to your SCM.

Whereas, a policy of automatically backing up everything except for your exclude list would have saved your bacon.

Re:Let's Stop Talking About "Backups" by spolsky on Monday December 14, 2009 @12:14PM (#994504)
What if you hit rm -rf * and that gets synched?

I don't mean to be rude, I don't know anything about you, but if you were a system administrator working for me, today would be your last day.

Re:Let's Stop Talking About "Backups" by jawngee on Monday December 14, 2009 @12:17PM (#994515)
Of course it would be his last day, he's a sysadmin and he just typed rm -rf * on /
Re:Let's Stop Talking About "Backups" by mechanical_fish on Monday December 14, 2009 @12:21PM (#994524)
You know, you tell yourself that you understand rm -rf. And yet somehow you always find a way to type it on something you shouldn't. Computers are perverse objects.

It's like reading an airline crash report, which I'm told often ends up sounding like a comedy of errors. Most airline crashes have an entire handful of causes, all of which are individually innocent, but on the one extremely rare occasion when they all happen at the same time they add up to disaster.

Re:Let's Stop Talking About "Backups" by ajross on Monday December 14, 2009 @12:49PM (#994591)
I fat-fingered the destination directory in a script once and accidentally shrunk my entire photo archive to 128x128 thumbnails. Thankfully I did have backups of all but a handful.

I'm always amazed at the number of people who spend their time thinking about replication strategies without understanding that data can be accidentally deleted in production too. I guess backups look "easy", so it's not as sexy an area of architecture planning.

Re:Let's Stop Talking About "Backups" by pyre on Monday December 14, 2009 @01:15PM (#994657)
People just don't always grasp the big picture. People that see RAID as a backup strategy are only looking at 'hard drive failure' as potential scenario. The same for machine replication. They are only trying to prevent against what happens if 'the machine dies.' As long as they realize that this doesn't protect against, "I accidentally deleted the files," then it's all good.
Re:Let's Stop Talking About "Backups" by mechanical_fish on Monday December 14, 2009 @06:41PM (#995436)
In a previous job I had a so-called system administrator laugh at my paranoia when I suggested that his awesome backup system -- protecting the equivalent of several dozen person-years of scientific data, millions of dollars' worth -- wouldn't protect us against a fire in the building. Or, for that matter, a thief who liked to steal computer hardware.

Job security note for sysadmins: When someone suggests a disaster scenario, don't open your response with a laugh.

Re:Let's Stop Talking About "Backups" by pyre on Monday December 14, 2009 @11:05PM (#995855)
Well, you know that you're talking to someone that doesn't plan for the future when, "What happens if the building burns down?" is responded to with, "You're just being paranoid." I guess the appropriate response would be to point out that using his logic tons of taxpayer money could be saved by nixing the fire departments (and tons of corporate money could be saved by not paying for fire protection -- alarms, detectors, escapes, etc).
Re:Let's Stop Talking About "Backups" by gaius on Monday December 14, 2009 @12:13PM (#994503)
if you can solve data synch between two sites, just keep your data synched.

LOL! And if you get a corrupt block on your primary site, what're you going to do? All your standbys are instantly tainted!

Better leave this one to the grownups.

Re:Let's Stop Talking About "Backups" by jws on Monday December 14, 2009 @12:18PM (#994516)
Replication is not backup. It only protects against hardware failure.

Spinning disks are good though. A lovely spot for backups. Just put them in a different building.

Edit: In the time it took me to write this 5 other people also lambasted this poor fellow. Ouch.

Re:Let's Stop Talking About "Backups" by InclinedPlane on Tuesday December 15, 2009 @01:58AM (#996079)
It doesn't matter if your data is replicated across 3 different continents. If all of your data is accessible behind the same admin login, or if it is all vulnerable to the corruption in the original, then it still has only 1 point of failure. A proper "spinning" backup is an immutable journaling system that can't be trimmed except via extremely secure methods (e.g. physical access to the machine).
Re:Let's Stop Talking About "Backups" by apowell on Monday December 14, 2009 @12:17PM (#994514)
A real-time mirror won't help if your data gets corrupted (via a fat-fingered shell command, errant script, or a compromised system). When it's appropriate, it's awesome, but I don't think it's a replacement for a static "offline" backup,
Re:Let's Stop Talking About "Backups" by InclinedPlane on Tuesday December 15, 2009 @01:54AM (#996076)
Sorry, this is not only WRONG but actually HARMFUL advice.

Redundancy is not a backup. If someone with full admin control to your system can destroy all of your data then you do not have backups. A proper backup is physically separate from your primary data and, preferably, can't be destroyed with mere admin access to the system. The number of sites that have had catastrophic data loss due to relying on mirroring instead of true backups is quite significant.