News for hackers. Stuff that matters.
Re:New tricks for XFS: support for subvolumes by amelius on Thursday March 01, 2018 @07:02PM (#16497583)
Basically copying files without actually copying them. For example, when making versioned backups, or when copying a large number of files from production to various development environments.

Something I wish there was a better solution for. Perhaps there is a filesystem out there that supports CoW in a way that fits this usecase (XFS perhaps even), but I haven't looked into it.

The disadvantage of using hardlinks is that you can't hardlink between users (user is a property of the file, not of the link to the file), and there's always the danger that a write takes place through one of the links. Imho, that should really be solved at the filesystem level using a CoW scheme.

Re:New tricks for XFS: support for subvolumes by josteink on Friday March 02, 2018 @02:32AM (#16499784)
> Basically copying files without actually copying them. For example, when making versioned backups, or when copying a large number of files from production to various development environments.

To me it sounds like you want simple snapshots and backups without the redundancy at the storage level.

So why don't use a filesystem which supports that natively, like ZFS or Btrfs?

Re:New tricks for XFS: support for subvolumes by mkj on Friday March 02, 2018 @04:23AM (#16500162)
Do they handle 64k snapshots?
Re:New tricks for XFS: support for subvolumes by lloeki on Friday March 02, 2018 @04:44AM (#16500236)
That's one backup every hour for 7+ years. I'm really curious about the use case.

Anyway, theoretically[0]:

> As you might know btrfs treats subvolumes as filesystems and hence the number of snapshots is indeed limited: namely by the size of files. According to the btrfs wiki the maximum filesize that can be reached is 2^64 byte == 16 EiB

But it seems practically you're hitting the mud at ~100 snapshots[1] but be sure to read the reply to that mail as it will depend on the use case and it might turn out to be fine way beyond that.

[0]: https://unix.stackexchange.com/questions/140360/practical-li...

[1]: https://www.mail-archive.com/[email protected]/msg...

Re:New tricks for XFS: support for subvolumes by KaiserPro on Friday March 02, 2018 @04:53AM (#16500269)
Not sure, I have ZFS as a docker storage backend, its yet to crash*

*not the best data point I realise....

Re:New tricks for XFS: support for subvolumes by e12e on Friday March 02, 2018 @01:31AM (#16499565)
> Perhaps there is a filesystem out there that supports CoW in a way that fits this usecase

I'd recommend zfs snapshots - or for something a bit different, maybe nilfs2:

http://www.linux-mag.com/id/7345/

Re:New tricks for XFS: support for subvolumes by koenigdavidmj on Thursday March 01, 2018 @07:07PM (#16497611)
Does btrfs, with cp --reflink, not meet your use case?
Re:New tricks for XFS: support for subvolumes by amelius on Thursday March 01, 2018 @07:10PM (#16497630)
Yes, thanks, I found that, but I'm not sure if I'm ready for btrfs yet. I have to look into it, and see if btrfs is sufficiently stable, etc.
Re:New tricks for XFS: support for subvolumes by caf on Thursday March 01, 2018 @09:38PM (#16498516)
reflinks should work on suitably recent XFS as well.
Re:New tricks for XFS: support for subvolumes by nialv7 on Friday March 02, 2018 @12:53PM (#16503947)
For versioned backup you should use reflink rather than hardlink.

Hardlink is not CoW. Modify the file all your backup will change too.