Re:How useful should copy_file_range be?
by webstrand on Friday February 19, 2021 @03:06PM
(#26197442)
Given all of the effort being put into zero-copy read/write in the kernel, I would assume there are significant performance gains available.
I suspect that some, correctly aligned, ranges could be copied with CoW semantics, thereby skipping the read/write altogether.
Re:How useful should copy_file_range be?
by bonzini on Friday February 19, 2021 @03:16PM
(#26197520)
On NFS you can avoid network traffic completely.
Re:How useful should copy_file_range be?
by jandrese on Friday February 19, 2021 @03:17PM
(#26197538)
Is this theoretical or is there support for it already in the kernel and NFS daemons?
Reading the manpage for copy_file_range the notes section states:
copy_file_range() gives filesystems an opportunity to implement "copy
acceleration" techniques, such as the use of reflinks (i.e., two or
more inodes that share pointers to the same copy-on-write disk blocks)
or server-side-copy (in the case of NFS).
But doesn't mention if these techniques are actually used. I guess there's some help in future proofing your code.Edit: I tested this on my Ubuntu 20.04 machine and a 1GB file full of random data sitting in the file cache. Using copy_file_range I could make a local-local copy in 0.595 seconds on average. Using a primitive copy/write loop took 0.600 seconds. But these values are somewhat noisy and the difference is down in the error margin. It doesn't appear that my 5.4.0 kernel on ext4 is employing the reflinks optimization.
Re:How useful should copy_file_range be?
by bonzini on Friday February 19, 2021 @03:19PM
(#26197558)
It's totally practical and already implemented by Linux on both the server and the client side.
Re:How useful should copy_file_range be?
by dallbee on Friday February 19, 2021 @04:05PM
(#26198022)
Can't share details, I've seen the copy_file_range optimization represent roughly 25% increased throughput for a prior employer. It's more than theoretical.
Re:How useful should copy_file_range be?
by the8472 on Friday February 19, 2021 @04:41PM
(#26198386)
ext4 has no reflink support. btrfs does by default, xfs under some configurations. server-side offload is supported by nfs and cifs but also needs server-side support.
Re:How useful should copy_file_range be?
by gravypod on Friday February 19, 2021 @03:30PM
(#26197678)
Even if it wasn't smart you'd still get a huge benifit from not context switching to/from kernel/user space for a standard copy implementation. Far fewer syscalls. There's a real performance benifit to be had there.
Re:How useful should copy_file_range be?
by bonzini on Friday February 19, 2021 @04:39PM
(#26198359)
ext4 does not support sharing blocks across multiple files, indeed. Try btrfs or perhaps xfs.
Re:How useful should copy_file_range be?
by 7786655 on Friday February 19, 2021 @03:04PM
(#26197422)
>The copy_file_range() system call looks like a relatively straightforward feature; it allows user space to ask the kernel to copy a range of data from one file to another, hopefully applying some optimizations along the way.
I can understand functions like sendfile() being able to cut down on context switches and being helpful for bulk data transfer, but is that the case here? How much benefit do you get from copy_file_range() vs. a read/write loop?
I do note with some amusement how the kernel developer basically went "Why are you using copy_file_range on things that aren't actually files?"