There is also Snebu (www.snebu.com, Github repo at github.com/derekp7/snebu). I wrote this about 8 years ago for my personal needs, because rsync / snapshot backups were running out of steam for me. What's different from Restic or Borg/Attic, is that the main "smarts" lives on the backup server (the Snebu binary), whereas the client side just uses find / tar to serialize the data. Backend side compresses, deduplicates, snapshots, handles a number of systems easily, and is fairly fast due to using SQLite for the metadata catalog.

A couple weeks ago I finished a client-side encryption module, got another weekend's worth of work to integrate encryption support into the back end. The client-side encryption module is already in the Git repository, but there is no documentation for it yet. My next project is a Web based GUI (maybe an Electron based client-side GUI too, depending on what makes sense).

Oh, and the most recent release supports granular user permissions, so you can grant a host (via an SSH account on the backup server) permissions to create backups, but not delete them. Or have an administrative user that can expire old backups but not read backups, for example. Can help with thwarting crypto viruses that try to delete backups.

> […] the main "smarts" lives on the backup server (the Snebu binary), whereas the client side just uses find / tar to serialize the data. Backend side compresses, deduplicates, snapshots […]

So am I right to infer that there is a ridiculous amount of overhead in the data transfer, as opposed to some of the other software mentioned? Would you use this over the internet? Or an untrusted network connection?

The communications is handled via ssh, so yes I would use it over the internet as the transfers are streaming. A full manifest (list of file names and metadata [file owner, mod time, size, etc]) is sent over the wire, and a return list of files that are needed to complete the snapshot (essentially changed / new files) is returned to the client. The client then tar's this smaller file list to stream to the remote server. Tar, in this case, is used to serialize the data, which gets extracted on the remote end (compressed and stored as sha1-named files [will be switching to sha-256 in the next major release]). Metadata gets put in the SQLite DB.

For data encryption (the main part is completed, but need to expand the backend to recognize encrypted data, should be finished shortly) -- the output of "tar" is piped through "tarcrypt". What tarcrypt does is it takes a standard tar file input, compresses/encrypts the file data, and outputs a tar file with some extended headers that contain info about the compression/encryption, including the RSA public key fingerprint used to encrypt the data, an HMAC, and the encrypted (passphrase-protected) private key (this can be made optional). The idea is that the encryption itself is AES-256-GCM, with a random key, which is encrypted with RSA public key. That way you can have encrypted backups without needing to have a password sitting in plain text on the client. And the RSA private key is passphrase encrypted, and sent along with the tar header to the server. On restore, you will be prompted (client-side) for the passphrase. This way you can restore a client even if the keyfile is destroyed.

I plan to make the encrypted key storage optional, but that would require that you manage the key file backup separately, and doesn't get you much more security (assuming you have an adequately strong passphrase).

Server requirements are a server with ssh access, and the snebu binary installed (optionally suid to a non-privileged backup user account, so that granular permissions can be employed for other accounts). And since Snebu is written in C, with only liblzo2, libcrypt, and sqlite2 as dependencies, it is easy to get it to work with a wide variety of systems (and the client side only requires a modern enough version of GNU "find" and "tar", unless encryption is used, which would require "tarcrypt" also -- modern in this case means withing the last 10 years, the "find" command needs to support -printf with the appropriate parameters).

With the exception of being able to upload to different targets (which can be piggybacked onto borg with something like rclone), borg is definitely the one with more whistles, not the other way around. Restic is lagging behind borg in most features, they always come up in borg first and then after a year restic maybe implements them. Important features currently missing in restic which exist in borg: compression, append-only backups, extensive include/exclude patterns, flagging any directory for exclusion directory exclude with a tag file inside it, setting "--chunker-params" - thus being able to adjust deduplication settings (like size of chunks, how hard it tries to look for duplicates etc) to the nature of the files and the limitations of RAM and CPU during backup. (and other minor ones).

Other differences which some might consider good and others bad include:

* Borg is in Python, Restic is in Go? Python is a more widely known language, the chance that more developers will be able to maintain it is higher, the chance for new features is higher, the chance that bugs will be found is higher. Both still provide a static binary download so deployment is very easy for both.

* Borg allows setting names for archives (which can still be templated with things like {now} or {username}), while Restic automatically names them with random ids. Borg's approach seems more user-friendly.

* Borg allows more choice in what encryption is used, including authenticated/non-authenticated, and including using no encryption at all (might be useful in some cases). (https://borgbackup.readthedocs.io/en/stable/usage/init.html#...)

* Borg allows specifying "max_segment_size = xxxx" on a repository, thus allowing to somewhat adjust the size of archive ("segment") files while they are stored on the server (default being 500 MB). Restic uses its own sizing and (if I remember correctly) more often than not it produces very small files (a couple of MBs) which crowd the filesystem. In general, archive file sizes can affect the filesystem and the uploading them to the cloud, where many cloud providers API operates on individual files, not on parts of them.

append-only backups: It supports this with the REST backend.

flagging any directory for exclusion directory exclude with a tag file inside it: It has this. It's called --exclude-caches and will exclude any directory with a "CACHEDIR.TAG" file in with the correct content.

Ok, that's certainly a good thing that it does support it now. Borg also allows to specify the name of that file, it does not restrict it to being "CACHEDIR.TAG". Giving an option to support both "CACHEDIR.TAG" and any other filename is a good thing. (One might want to exclude other directories that way, unrelated to cache directories.)

CACHEDIR.TAG seems to be based on a spec that multiple pieces of software are starting to use. Give it a quick google.

I can't think of any reason why you would need to customise the name of the file, but I'm sure there are use cases out there.

This is also a problem.

I might like to mark certain things as not backed up because they contain sensitive or otherwise unrelated information. But I might not want other tools treat them as cache data, that is, transient and safe to delete.

Then you can just add `--exclude filename` to your backup job.

Actually, "--exclude-caches" in restic is just an alias equivalent to the following [1]:

--exclude-if-present "CACHEDIR.TAG:Signature: 8a477f597d28d172789f06886806bc55"

You can specify the tag filename, and optionally append ":initial content" (the tag file must start with this content to avoid false tagfile matches).

[1] https://restic.readthedocs.io/en/stable/040_backup.html#incl...

Like I wrote in the original comment, borg's additional functionality is that it supports both CACHEDIR.TAG and any other filename if that is what user wants.

As I wrote in the previous comment, you can do that in restic with the "--exclude-if-present" option where you can specify whatever filename you want as well as optionally some content it must start with to avoid false positives.

Now I finally get it, thank you :). So Borg and Restic are basically on par considering that feature after all!

> Borg is in Python, Restic is in Go? Python is a more widely known language

One doesn't need an interpreter + supporting libraries to run a backup program. With a Go program one just drops a statically compiled binary in $PATH and it just works.

Anyway, I'm still using duplicity. It backups on rsync, ftp, you name it. Written in Python as well, unfortunately. Don't care abot S3 etc as I don't like loss of control over my backups.

To repeat my original post, both tools provide a static binary download, making it easy to deploy. Both "just work". It's a single file that you chmod and run.

Duplicity, as you may know is not a deduplicating backup solution, it is an incremental backup solution, suffering from all the flaws and limitations thereof. If you happened to not realize the difference, I highly recommend this short but informative blog post from Backblaze: https://www.backblaze.com/blog/backing-linux-backblaze-b2-du...

Static binary as the whole interpreter and modules bundled in a fat binary? No thanks.

I know the difference, but this way we have 4G volumes we could eventually burn to DVDs if we'd want to. No backup repositories, cloud storage solutions or new and obscure network protocols involved. Deduplication is also uninteresting for us because the largest part of our backup are SQL dumps. This is why we make full backups every time.

You don't have 14 MBs to spare for a static binary? What is this zealotry against interpreter languages? You know that your Go static binaries are basically the same thing - one binary packaged with all the bundled modules into a fat file? Where is the logic here?

Also why would deduplicating be uninteresting when the backups are SQL dumps? That is EXACTLY the scenario in which deduplication would shine, because it would take each dump and find parts of it that are the same as the latest backup, finding the individual small chunks that have changed in those dumps, and only backing them up. Do you actually know how deduplication works? You might want to look that up more properly.

No zealtory. I'd rather have the interpreter installed from the distribution and drop the fatpacked program with bundled libraries and everything in a bin directory. If it needs installing on Windows then the fatcpacked binary with bundled interpreter is also fine.

> With a Go program one just drops a statically compiled binary in $PATH and it just works.

Until one of the statically compiled dependencies needs a security fix.

Then people realize why shared objects where invented.

That's not why they were invented, and it's not even a good reason to use them.

I would also say that Borg is more feature-complete and mature, not the other way round.

If you don't want to run your own server, I offer a hosted Borg Backup service: https://www.borgbase.com

This would give you some Borg-specific features, you can't easily get with your own VPS, like monitoring for outdated backups and restricting certain keys to append-only mode.

Borg's encryption is questionable at best. Performance wise it's not particularly good and probably will never get better because of the large, somewhat complex Python codebase.

What's questionable about it? It seems that they use modern AEAD ciphers in a reasonable way.

I'm more concerned about the repository format and config file (i.e. attack surface, since the repo is potentially untrusted).

Performance is actually better than Restic, and performance-critical parts of Borg are written in C or use C libraries.

> What's questionable about it? It seems that they use modern AEAD ciphers in a reasonable way.

No, using one key per repository and a persistent message counter is not a reasonable design.

https://borgbackup.readthedocs.io/en/stable/internals/securi...

For what threat model does this matter?

"When the above attack model is extended to include multiple clients independently updating the same repository, then Borg fails to provide confidentiality (i.e. guarantees 3) and 4) do not apply any more)."

Edit: I've posted this a bunch of times here, pretty much every time it caught my eye when someone said this tool has good crypto, and by now I'm used to people just downvoting it and saying it doesn't matter because obviously no one ever would use it like that and the design is fine etc. (isn't the point of deduplication to save disk space?)

From OPs link:

> If we perform similar backups to the same remote destination over SSH (Borg) and SFTP (Restic), the initial backups take roughly the same time, but the subsequent incremental backups take something like 10x longer with Restic.

Also I'm wary of encrypting personal backups. I think the chances of me forgetting a password that I used literally once are quite high, and what's the point of a backup if you can't actually access it?

I'm worried about that too, but you can deal with it by making the system that automates your backups also test your memory of the password (while still creating a non-interactive backup even if you ignore the prompt). Alternatively, reuse a password that you regularly use offline, like a disk encryption or SSH passphrase.

Or use a password manager..

I would also like to know more about why you think Borg's encryption is questionable. I'm not familiar with Borg's internals, but there's not much one could do wrong here, as it should just be simple symmetric encryption. Does it not use a strong cipher? Is the cipher used incorrectly? Is the key derivation method weak?

restic is simpler than borg, because it doesn't require a special borg server (it just writes out to files - or block storage).

This is somewhat misleading: borg absolutely does not require its server software to be installed on the target server. It supports (and works well) with a simple sshfs mount.

This is important -- it means that restic can trivially be used with e.g. S3.

I've loved using restic sofar. I've got about 3TB of stuff backed up to s3 using the restic cli. It is simple and just works.

If anyone though knows how I can more easily restore a file that would be great. I have to supply the restoration snapshot ID right now, and I'd rather just do 'latest' and have it find the newest version of the file in all of the snapshots and restore it. Is this possible ?

Something like:

   restic restore -i '/path/to/file.txt' -i . latest

instead of:

   restic restore -i '/path/to/file.txt' -i . search-file-snapshot-list-for-big-sha-here

you could always mount the entire backup, using

    restic mount backup/

and pick the file(s) you are looking for from the `backup/snapshots/latest/` directory.

It also means that the sending client has to all the work. restic is enormously resource hungry compared to borg. It can be hit by the OOM killer on small VPSs.

I use borg on a small VPS that hosts my side project to get free backups.

Borg runs doing incremental backups to a local directory. I sync the borg backup folder to a free BackBlaze B2 bucket. The whole thing comes in around 5GB of backup including database backup and all the hosted files and configuration files.