News for hackers. Stuff that matters.
Re:What I Would Do If I Ran Tarsnap by tomp on Thursday April 03, 2014 @02:09PM (#7524774)
Data deduplication, incremental backups.
Re:What I Would Do If I Ran Tarsnap by tptacek on Thursday April 03, 2014 @03:14PM (#7525307)
HEAD-DESK.

Deduplication and incremental backups are table-stakes for backup software.

The reason a business would use Tarsnap rather than some other backup service is the level of confidence that Colin can provide that Tarsnap will reliably protect their data from attackers, including compelled insiders at Tarsnap.

In other words, Tarsnap can offer an enterprise an offsite backup service that is demonstrably as safe as backup data that the enterprise retains direct custody of.

That is not an offering other backup providers can reliably duplicate.

Re:What I Would Do If I Ran Tarsnap by tomp on Thursday April 03, 2014 @03:34PM (#7525642)
That's right, I was just answering the parent what advantages Tarsnap it has compared to a OSS, bash-pipe-made, tar+encrypt solution.
Re:What I Would Do If I Ran Tarsnap by tptacek on Thursday April 03, 2014 @03:42PM (#7525740)
Security remains the most important difference between those two options.
Re:What I Would Do If I Ran Tarsnap by robryk on Thursday April 03, 2014 @04:20PM (#7526231)
I assume you refer to all the seemingly nitty problems with the pipeline above (from what I can see, there is no way to verify that the archive wasn't tampered with).

Would you say the same about a solution that signs and encrypts the archive with gpg (signs with a machine's key and encrypts it to the owner's key). If so, can you elaborate on some examples of security problems that solution could have?

Re:What I Would Do If I Ran Tarsnap by tptacek on Thursday April 03, 2014 @04:24PM (#7526274)
Are you asking if I could design you a secure backup system?

I could, and it might asymptotically approach the quality of Colin's.

I don't think you're comfortable with the amount of money I'd charge for that service.

You're better off paying Colin cost-plus for AWS storage, since that's all he seems to want to charge. :)

Re:What I Would Do If I Ran Tarsnap by Nanzikambe on Thursday April 03, 2014 @02:20PM (#7524873)
Heh apologies, my fault for trying to be clever, the mechanism I actually use is incremental and deduplicated. I substituted it for tar to simplify.

I actually use ZFS (filesystem), so my backup flow is closer to:

  TSTAMP="backup-$(date "+%Y%m%d-%H%M%S")"
  zfs snapshot -r $TSTAMP
  zfs send $TSTAMP | \
      xz -z | \
      openssl enc -aes-256-cbc -e -salt | \
      > /mnt/your/networked/google/drive/backup.$(hostname -a).$TSTAMP.aes.tar.xz
The underyling ZFS filesystem is deduplicated at filesystem level, and snapshots are incremental. THere're a few other minor differences (the dest is another ZFS host which syncs to Google drive, and I nuke the local snapshot after send because RAID 1+0 space is more expensive than RAID1 .. )
Re:What I Would Do If I Ran Tarsnap by Nanzikambe on Thursday April 03, 2014 @02:29PM (#7524963)
To answer my own question: deduplication :)

I had not considered multiple backup sources, mine is deduplicated per host, am I to understand tarsnap is deduplicated across all hosts sharing a set of keys?

Re:What I Would Do If I Ran Tarsnap by vbit on Friday April 04, 2014 @12:04AM (#7529217)
Also, easier restore and snapshot deletion.

Consider how you would restore using incremental ZFS snapshots. You'd have to pull all the snaphots, unpack the base snapshot and then sequentially unpack each incremental snapshot.

In tarsnap, the server will compute the 'snapshot' you want for you, and will only send you the data blocks that belong to that snapshot.

In tarsnap, you can also delete any snapshot you want, and only blocks belonging exclusively to that snapshot will be deleted. In your system, deleting a snapshot means you lose all snapshots from that one until the next full snapshot.

Also, in ZFS you're limited to backing up complete datasets, but with tarsnap you can backup any set of files you want.

Re:What I Would Do If I Ran Tarsnap by beagle3 on Thursday April 03, 2014 @04:30PM (#7526327)
I think that's the case.

Note: https://github.com/bup/bup does that too (though it does not encrypt), and http://liw.fi/obnam/ does too (and it does encrypt).

What tarsnap gives you that obnam doesn't is (a) managed cloud storage, (b) tarsnap's history and reputation, and (c) Colin's personal reputation. That's a lot, and it costs money above the S3 storage costs (which you could point obnam at).

Re:What I Would Do If I Ran Tarsnap by foobarqux on Thursday April 03, 2014 @03:57PM (#7525927)
I would like to do something similar with BTRFS.

How are your snapshots incremental? In BTRFS you would need to specify a base snapshot.

What is the restore process? You init a zfs file systems and then zfs receive the backups in chronological order? How are the dependencies between snapshots managed?

Re:What I Would Do If I Ran Tarsnap by vbit on Thursday April 03, 2014 @11:56PM (#7529168)
The 'zfs send' command will send an incremental snapshot if you specify '-i snap1 snap2'.

To restore, of course, you'll have to have snap1, and then you can apply the increment.

Re:What I Would Do If I Ran Tarsnap by foobarqux on Friday April 04, 2014 @01:29PM (#7532936)
That's what I thought. I wanted to know if the OP had some way to manage the dependencies between snapshots.