"For a simple tool, use rsnapshot ..."

Even simpler is to do a dumb, 1:1 mirror, with rsync and then rotate zfs snapshots on the server side.

Two benefits here over rsnapshot (or rsync snapshots, with hardlinks) are:

1) rsync snapshots diff on a file granularity - if you change one bit of the file, your penalty is the storage space for that entire file. ZFS, on the other hand, diffs on a bits level so your penalty is just the changed portions of that paticular file.

2) Depending on your implementation, ZFS snapshots are immutable/read-only ... so neither a misconfiguration nor ransomware/mallory can destroy those backups if you lose your credentials.

Historically, people would create and maintain "rsync snapshots" inside their rsync.net accounts using 'cp -al' and other unix commands run over SSH.[1]

After transitioning to ZFS, however, we encourage all customers to just do a "dumb" 1:1 rsync to us and let us maintain and rotate their preferred snapshot schedule (days/weeks/months/quarters/years).

[1] https://www.rsync.net/resources/howto/remote_commands.html

Just curious, how do you sync file ownership and permissions when going to a ZFS target server with a non privliged account? Or would you have all the files on the target owned by the user ID accessing the target server, then keep the metadata (owner/mode/SE Linux attributes / ACLs, etc) in a separate log file?

And do you typically turn on ZFS compression? I tried that years ago with ZFS under Linux, but had major slowdowns after a while. But I'm sure it has improved quite a bit in more recent years. Also how does ZFS stack up against Btrfs in your experience?

But for a direct-to-cloud backup, the way you do it is really great -- to me it is much easier than messing with things like S3 buckets and similar technoligies.

Well, if you are using a sophisticated tool like restic or borg or duplicity, the file attributes are bundled up into the encrypted chunks and all is well.

However, if you are using plain old rsync, you can use the --fake-super command line switch to maintain those attributes and I believe rsync.net customers are using that successfully.

Yes, you are correct - in the old days (2001-2010 or so) we maintained a pair of "permission scripts" which allowed customers to dump all their attributes to a file and then reapply it later. Again, --fake-super seems to "fix" all of this.

Generally speaking, you should always enable ZFS encryption. It improves many aspects of ZFS behavior and performance - not just space usage. Unless you have very bizarre workloads or hardware, I think that in 2020 there is no reason to not enable compression. Deduplication is another story and is very expensive to maintain - rsync.net has never run de-dupe on any systems.

We (rsync.net) and I (personally) have zero experience with btrfs ...

That shold be "you should always enable ZFS compression"?

Oh, sorry - yes, you are correct:

s/encryption/compression

That was actually the original inspiration for Snebu (not rsnapshot specifically, but a custom script implementing similar functionality based on the original rsync snapshot paper). Snapshots are so much better than level-based backups (full/differential/incremental) when using disk-backed storage.

I ended up running into this problem: https://news.ycombinator.com/item?id=8305283 when I got to 20 hosts and a year's worth of backups (14 daily, 6 weekly, and 12 monthly snapshots, so 32 snapshots per hosts). It was made worse in my case because I was using ZFS with compression. So the original implementation of Snebu used shell scripts, flat files for the catalog, and a small utility to burst apart tar files.