News for hackers. Stuff that matters.
Re:Restic – Backups Done Right by Erlich_Bachman on Thursday October 31, 2019 @03:04PM (#21412406)
Another major problem is that it does not support compression: all files are stored uncompressed on the backup server (even though they are deduplicated) (sic).
Re:Restic – Backups Done Right by mike-cardwell on Thursday October 31, 2019 @03:24PM (#21412610)
You know, I actually assumed it had compression. It looks like it will get it eventually though, so I'm not too worried. That said, I can get some benefits by compressing some of my backups myself. For example, I currently backup my postgres dbs with cron jobs like:

$ pg_dump dbname | restic backup --stdin --stdin-filename dbname.dump --tag dbname -q

With restoring as simple as:

$ restic dump latest --tag dbname dbname.dump | psql dbname

I should probably stick a gzip/gunzip between those two commands.

Re:Restic – Backups Done Right by adrianmonk on Thursday October 31, 2019 @03:51PM (#21412950)
If you stick a cheese up in there, restic will probably lose some of its ability to deduplicate.

If it does compression, then it's not an opaque process to it. But if you do it, you are essentially obfuscating the structure of the data.

Re:Restic – Backups Done Right by tatersolid on Thursday October 31, 2019 @10:37PM (#21415821)
gzip with the —rsyncable flag will still allow for deduplication. It uses a content-aware rolling checksum and resets the compression dictionary whenever that checksum has a certain number of zeros.

The result is a valid gzip file with a few percent less compression, but friendly to deduplication (which is what rsync does).

Re:Restic – Backups Done Right by mike-cardwell on Thursday October 31, 2019 @03:58PM (#21413043)
Good point. I'll stay away from cheesing up my commands and wait for it to be built in.
Re:Restic – Backups Done Right by mynegation on Thursday October 31, 2019 @04:41PM (#21413519)
What is "cheesing up"?
Re:Restic – Backups Done Right by adrianmonk on Thursday October 31, 2019 @10:12PM (#21415722)
It's what Google speech-to-text thinks I meant when I said "gzip" into my phone.
Re:Restic – Backups Done Right by sliken on Friday November 01, 2019 @04:47AM (#21417109)
Ha, I was thinking there's a new gzip competitor I needed to track down called cheese.
Re:Restic – Backups Done Right by eps on Thursday October 31, 2019 @06:27PM (#21414430)
If this is a concern, why not delegate this to the file system level - just stick backups on a compressed volume and be done with it, no?
Re:Restic – Backups Done Right by Erlich_Bachman on Thursday October 31, 2019 @06:38PM (#21414507)
No.

You cannot stick a compression step before restic, because it will mess up the deduplication - the chunker will not find the same chunks, and will not be able to find out which files have same contents (not as easily at least).

And you cannot stick it after restic, compressing the backup archives (or storing them on a compressing filesystem), because those are encrypted, and any encrypted file has high entropy and does not compress. (It doesn't help that restic makes the encrypotion mandatory).

So no, compression really needs to be an integral step in a backup system. Restic is definitely lacking in this regard.

Re:Restic – Backups Done Right by eps on Friday November 01, 2019 @02:36AM (#21416654)
Ah, you are right. I forgot it was encrypting the backups.
Re:Restic – Backups Done Right by BurningCycles on Thursday October 31, 2019 @06:13PM (#21414311)
Yes, that is the one thing that keeps me from using Restic as my backup solution.

Unfortunately it does not seem like it's a prioritised feature given that it has been planned since late 2014.