News for hackers. Stuff that matters.
Re:Proxmox VE 6.2 by icefo on Tuesday May 12, 2020 @08:31AM (#23153208)
I love Proxmox but the absence of incremental backups is very annoying. If you have a large vm whose contents don't change much your options are limited as storage (edited: and bandwidth) is not infinite.

I also don't want to use aryufan patches to enable incremental backup. I feel like they could probably partner with duplicacy to ship that feature. Duplicacy already released a paid version specifically designed to backup VMware hosts so it should be possible to adapt it to backup Proxmox hosts.

(Thank you all for your comments ! They gave me some ideas)

Re:Proxmox VE 6.2 by beagle3 on Tuesday May 12, 2020 @09:05AM (#23153518)
Use a CoW/snapshotting backing file system for your backup (btrfs/zfs), or one of the modern de-duplicating backup systems (bup/borg/restic), and the space taken will be proportional to the changed data, while still providing random-access to any snapshot -- Not much more you can expect in terms of space usage.

snapshotting filesystems will be much more efficient in terms of time/effort compared to de-duplicating later, but other than that, all of these solutions are mostly equivalent.

Re:Proxmox VE 6.2 by icefo on Tuesday May 12, 2020 @09:22AM (#23153672)
I don't argue that it would work but when you do that you're still sending all the data over some data link.

For example if I have 1.5TB of VM and only 10-20Gb changed today I'm still sending everything to a remote storage. Snapshotting locally and sending the zfs volume seems to be a more efficient solution

Re:Proxmox VE 6.2 by beagle3 on Tuesday May 12, 2020 @09:34AM (#23153780)
That's not at all true for bup and borg, almost certainly not true for restic (Haven't used it myself and can't tell for sure).

bup and borg (and likely restic) keep local hash summaries that are synchronized with the repository if it is remote. They would still have to _read_ the entire 1.5TB VMS if every vm file has changed (there are no facilities on a regular file system that they can use to figure out which parts changed), but they WILL use those local hashes to find out parts actually changed, and only send those changes to the remote storage. The local hashes typically occupy 0.5% of the data size (for bup, by default: 20 byte SHA1 per 8192 bytes on average;

You can change the chunking parameters for bup/borg/restic for a tradeoff between cache size and granularity of change detected; for VMs it might make more sense to e.g. have 256KB chunks in which case you'll have 0.01% local storage overhead, and likely still negligible network transfer overhead.

> I don't argue that it would work but when you do that you're still sending all the data over some data link.

That's only true if you include e.g. the SATA link in that list; To avoid that, you need a filesystem that effectively tracks change ("damage") regions, or VM storage that uses lots of underlying files so you can track it on a regular file system (parallels on the mac used to do that so time machine backups are efficient). no way around it.

Re:Proxmox VE 6.2 by toomuchtodo on Tuesday May 12, 2020 @11:24AM (#23154954)
> there are no facilities on a regular file system that they can use to figure out which parts changed

Doesn't ZFS do block level checksums that could be used for this, and backup tooling could use block pointer metadata for this purpopse?

Re:Proxmox VE 6.2 by beagle3 on Tuesday May 12, 2020 @11:29AM (#23155004)
Yes, I mentioned that you need a file system that can do that (afaik, zfs, btrfs on Linux, and hammer2 on dragonfly) Or that you’ll need to read it all.
Re:Proxmox VE 6.2 by amarshall on Tuesday May 12, 2020 @09:34AM (#23153793)
This should be able to be avoided if the backup target is also ZFS by using `zfs send`.
Re:Proxmox VE 6.2 by iggldiggl on Tuesday May 12, 2020 @08:50AM (#23153372)
I've been wondering about the very same problem (and stumbled across aryufan's patches as well), and discovered that they're now actually working on an expanded backup solution of their own:

https://git.proxmox.com/?p=proxmox-backup.git;a=tree

https://forum.proxmox.com/threads/proxmox-backup-server-and-...

Re:Proxmox VE 6.2 by JustFinishedBSG on Tuesday May 12, 2020 @08:35AM (#23153252)
Use ZFS as backend and tada your problem stops existing
Re:Proxmox VE 6.2 by icefo on Tuesday May 12, 2020 @08:51AM (#23153388)
I could only use snapshots and regularly send the pool somewhere but it feel a bit hacky
Re:Proxmox VE 6.2 by jblwps on Tuesday May 12, 2020 @11:30AM (#23155018)
You don't send the whole pool--you send the datasets (ZFS filesystems) that you're concerned with.
Re:Proxmox VE 6.2 by blue1 on Tuesday May 12, 2020 @09:10AM (#23153556)
pve-zsync ?