News for hackers. Stuff that matters.

A comment on How is GNU `yes` so fast?. Parent comment

Re:How is GNU `yes` so fast? by mrb on Tuesday June 13, 2017 @03:52AM (#14543425)
Actually yes should use vmsplice().

And pv on the other side of the pipe should use splice().

Now that would be a complete zero-copy I/O path, purely limited by the CPU, not by memory bandwidth. It would benchmark at hundreds of GB/s :)

[ Parent | on HN ]
Re:How is GNU `yes` so fast? by gpderetta on Tuesday June 13, 2017 @08:14AM (#14544197)
Someone posted an implementation which uses vmsplice on the reddit thread:

https://www.reddit.com/r/unix/comments/6gxduc/how_is_gnu_yes...

[ Parent | on HN ]
Re:How is GNU `yes` so fast? by mrb on Tuesday June 13, 2017 @08:44AM (#14544317)
His code can further be optimized by having only 1 large element in the iovec (vs. many 2-byte "y\n" elements).
[ Parent | on HN ]
Re:How is GNU `yes` so fast? by nyanscat on Tuesday June 13, 2017 @09:46AM (#14544647)
from what i can see it's already in a large iovec buffer
[ Parent | on HN ]
Re:How is GNU `yes` so fast? by mrb on Tuesday June 13, 2017 @09:49AM (#14544659)
Correct, I misread the code. However bumping the pipe size buffer bumps the speed from 28 to 74 GB/s on my Skylake CPU (i5-6500).

Edit: I optimized further and now get 123 GB/s See https://www.reddit.com/r/unix/comments/6gxduc/how_is_gnu_yes...

[ Parent | on HN ]
Re:How is GNU `yes` so fast? by gpderetta on Tuesday June 13, 2017 @12:24PM (#14545981)
next step is playing tricks with memory mapping so that the buffer is large in virtual memory but only takes a single page in physical memory, fitting in L1 (at least on the read side).
[ Parent | on HN ]
Re:How is GNU `yes` so fast? by qb45 on Tuesday June 13, 2017 @06:01AM (#14543797)
> And pv on the other side of the pipe should use splice().

Good catch, I just noticed that it does.

[ Parent | on HN ]