News for hackers. Stuff that matters.
Re:Encrypted Backup Shootout by chenxiaolong on Saturday January 02, 2021 @10:54PM (#25619262)
restic used to be my tool of choice in the past, but I unfortunately ran into an issue where a single corrupted blob caused many files to be unrecoverable. Thankfully, I noticed it when testing the backups so I didn't actually lose any data.

I've since switched to using encrypted rclone backups due to the design decision of having each source file map to one target file. For my specific use case, it's more important that corruption of backup files has minimal impact vs. having good deduplication. I also like that the format is simple enough where I feel confident I could write an extraction tool that skips checksum validation or bad blocks if I really need to.

Re:Encrypted Backup Shootout by andrewchambers on Saturday January 02, 2021 @11:12PM (#25619367)
Yeah, this is one downfall of these tools, they amplify the risk of corruption unless you take additional redundancy measures.
Re:Encrypted Backup Shootout by StavrosK on Sunday January 03, 2021 @12:05AM (#25619630)
Did you test the backups with Restic's built in test command, or by restoring?
Re:Encrypted Backup Shootout by chenxiaolong on Sunday January 03, 2021 @06:15PM (#25626207)
I tested by attempting to do a full restore to a temp directory.
Re:Encrypted Backup Shootout by stavros on Sunday January 03, 2021 @08:39PM (#25627101)
Do you remember if `restic check` said that everything was fine?
Re:Encrypted Backup Shootout by chenxiaolong on Monday January 04, 2021 @01:26PM (#25635698)
Unfortunately I don't remember. From the (little) notes I kept, the corruption occurred with restic version 0.9.3. I had also tried restoring with the latest code from git master, but I didn't make a note of the exact commit (it didn't make a difference anyway).

The commands that triggered the corruption were the following. I tested the restore around the 10th run of this.

    repo=sftp:chenxiaolong@<ip address>:/home/restic
    restic -r "${repo}" backup -x /home
    restic -r "${repo}" backup -x --exclude /stuff/Android/AOSP /stuff
    restic -r "${repo}" forget --keep-last 3 --prune
Re:Encrypted Backup Shootout by aborsy on Sunday January 03, 2021 @01:43AM (#25620048)
Does this risk exist with un-encrypted Borg back ups also?

(Compressed deduped also doesn’t map file to file).

Re:Encrypted Backup Shootout by dimatura on Sunday January 03, 2021 @10:18AM (#25622215)
Almost certainly, yes. It's an inherent risk with any backup system that deduplicates at at a block level, encrypted or not -- if the same block appears in many files, then it's stored only once (which is good in terms of space efficiency), but if that block is corrupted, then all the files sharing that block will be affected. Incidentally, this is not just an issue with backup systems; almost any compressed data format will also have similar risks. It's a tradeoff of the safety in redundancy for space efficiency.
Re:Encrypted Backup Shootout by jbboehr on Sunday January 03, 2021 @09:12PM (#25627272)
Would it be possible or desirable for one of these backup system to natively implement parity?