Re:Encrypted Backup Shootout
by andrewchambers on Saturday January 02, 2021 @11:12PM
(#25619367)
Yeah, this is one downfall of these tools, they amplify the risk of corruption unless you take additional redundancy measures.
Did you test the backups with Restic's built in test command, or by restoring?
Re:Encrypted Backup Shootout
by chenxiaolong on Sunday January 03, 2021 @06:15PM
(#25626207)
I tested by attempting to do a full restore to a temp directory.
Do you remember if `restic check` said that everything was fine?
Re:Encrypted Backup Shootout
by chenxiaolong on Monday January 04, 2021 @01:26PM
(#25635698)
Unfortunately I don't remember. From the (little) notes I kept, the corruption occurred with restic version 0.9.3. I had also tried restoring with the latest code from git master, but I didn't make a note of the exact commit (it didn't make a difference anyway).
The commands that triggered the corruption were the following. I tested the restore around the 10th run of this.
repo=sftp:chenxiaolong@<ip address>:/home/restic
restic -r "${repo}" backup -x /home
restic -r "${repo}" backup -x --exclude /stuff/Android/AOSP /stuff
restic -r "${repo}" forget --keep-last 3 --pruneDoes this risk exist with un-encrypted Borg back ups also?
(Compressed deduped also doesn’t map file to file).
Almost certainly, yes. It's an inherent risk with any backup system that deduplicates at at a block level, encrypted or not -- if the same block appears in many files, then it's stored only once (which is good in terms of space efficiency), but if that block is corrupted, then all the files sharing that block will be affected. Incidentally, this is not just an issue with backup systems; almost any compressed data format will also have similar risks. It's a tradeoff of the safety in redundancy for space efficiency.
I've since switched to using encrypted rclone backups due to the design decision of having each source file map to one target file. For my specific use case, it's more important that corruption of backup files has minimal impact vs. having good deduplication. I also like that the format is simple enough where I feel confident I could write an extraction tool that skips checksum validation or bad blocks if I really need to.