News for hackers. Stuff that matters.
Re:Rsync.net: ZFS Replication to the cloud is here and fast by LeoPanthera on Friday December 18, 2015 @11:16AM (#10759192)
Yes, but after the data has changed, I don't want to waste time sending data in the snapshot that is no longer in the live data. I'd rather just carry on from where it is now. My current system does this just fine.
Re:Rsync.net: ZFS Replication to the cloud is here and fast by masklinn on Friday December 18, 2015 @11:31AM (#10759263)
So you add a large amount of data, you snapshot, you delete that data and at that point you send the snapshot but don't want to send the deleted data? Why not just re-snapshot and send the new snapshot instead of the old one?

> And I'm also unsure whether it's even possible interrupt a transfer and resume it later, if the underlying data has changed

There is no "underlying data has changed" in a snapshot, it's a revision of the filesystem. As for interrupting and resuming later, Delphix apparently has contributed a resumable zfs send/receive to OpenZFS, though I don't know if it's available in yours or rsync.net's distribution: http://blog.delphix.com/matt/2015/03/25/resumable-zfs-sendre...

As for the concerns of sending small files first, I don't think ZFS could even use that: I don't believe it's content-addressed, so if you send a set of blocks from snapshot 1, interrupt and send snapshot 2 instead the blocks common to both will still be re-sent, ZFS only knows the block commonality from snapshots and since snapshot 1 hasn't been pushed its contents can't be used for snapshot 2. Unless the resumable replication support uses content addressing but that doesn't seem to be the case.

Re:Rsync.net: ZFS Replication to the cloud is here and fast by LeoPanthera on Friday December 18, 2015 @02:03PM (#10760240)
> Why not just re-snapshot and send the new snapshot instead of the old one?

Because then you have to start form scratch every time. The snapshots are so large that if you wanted to send a complete one, a large amount of stuff will have changed.