News for hackers. Stuff that matters.
Re:Axboe Achieves 8M IOPS Per-Core with Newest Linux Optimiz... by egberts1 on Tuesday October 19, 2021 @07:09AM (#28916858)
True. But, but, … BUT when it comes to files found on differing (physical/logical/virtual) disk partitions, ye gonna need that block copying mechanism within ‘cp’.
Re:Axboe Achieves 8M IOPS Per-Core with Newest Linux Optimiz... by kerneltrap on Tuesday October 19, 2021 @04:10PM (#28923104)
Not necessarily, while reflink works only within the boundaries of the same filesystem there are other optimizations that can operate across filesystems (even networked ones). For example copy_file_range() [1] allows for copy offloading (depending on the underlying fs/storage), which was incidentally also added to coreutils v9.0 [2].

There are also optimizations on other layers, e.g. transparent compression (e.g. btrfs) and online block-level dedup (e.g. dm-vdo), that can aid in minimizing and/or avoiding actual block copies.

When all else fails, cp will anyway fallback to buffered IO, so it won't be bound by the rate of IO (as long as writeback can keep up with memory pressure).

Of course improvements on all layers are always welcome and there are certainly workloads that can benefit from an optimized raw throughput, low-latency and scalability. Demanding workloads are typically using dedicated IO stacks and bypass the kernel altogether (e.g SPDK), but the amazing efforts of Axboe and everyone else in the kernel community have been continuously bringing linux up to par in terms of IO performance.

[1] https://man7.org/linux/man-pages/man2/copy_file_range.2.html [2] https://git.savannah.gnu.org/cgit/coreutils.git/commit/?id=4...

Re:Axboe Achieves 8M IOPS Per-Core with Newest Linux Optimiz... by egberts1 on Saturday October 23, 2021 @07:52PM (#28973545)
Jen Axboe Rocks!