News for hackers. Stuff that matters.
Re:How fast are Linux pipes anyway? by themulticaster on Thursday June 02, 2022 @09:02AM (#31594309)
If you ever need to write a random character to a pipe very fast, GNU coreutils has you covered with yes(1). It runs at about 6 GiB/s on my system:

  yes | pv > /dev/null
There's an article floating around [1] about how yes(1) is extremely optimized considering its original purpose. In care you're wondering, yes(1) is meant for commands that (repeatedly) ask whether to proceed, expecting a y/n input or something like that. Instead of repeatedly typing "y", you just run "yes | the_command".

Not sure about how yes(1) compares to the techniques presented in the linked post. Perhaps there's still room for improvement.

[1] Previous HN discussion: https://news.ycombinator.com/item?id=14542938

Re:How fast are Linux pipes anyway? by dpflug on Thursday June 02, 2022 @09:30AM (#31594625)
Faster still is

  pv < /dev/zero > /dev/null
Re:How fast are Linux pipes anyway? by BenjiWiebe on Thursday June 02, 2022 @09:38AM (#31594710)
Yes but you don't have control of which character is written (only NULLs).

yes lets you specify which character to output. 'yes n' for example to output n.

Re:How fast are Linux pipes anyway? by rocqua on Thursday June 02, 2022 @10:41AM (#31595480)
Yes doesn't just let you choose a character. It lets you choose a string that will be repeated. So

    yes 123abc
will print

    123abc123abc123abc123abc123abc
and so on.
Re:How fast are Linux pipes anyway? by jolmg on Thursday June 02, 2022 @06:13PM (#31601200)
each time terminated by a newline, so:

  123abc
  123abc
  123abc
  ...
Re:How fast are Linux pipes anyway? by gitgud on Thursday June 02, 2022 @10:05AM (#31595039)
> It runs at about 6 GiB/s on my system...

Honest question: what are the practical use cases of this?

Repeatedly typing the 'y' character into a Linux pipe is surely not that common, especially at that bit rate. Also seems like the bottleneck would always be the consuming program...

Re:How fast are Linux pipes anyway? by linsomniac on Thursday June 02, 2022 @10:21AM (#31595254)
Historically, you could have dirty filesystems after a reboot that "fsck" would ask an absurd number of questions about ("blah blah blah inode 1234567890 fix? (y/n)"). Unless you were in a very specific circumstance, you'd probably just answer "y" to them. It could easily ask thousands of questions though. So: "yes | fsck" was not uncommon.
Re:How fast are Linux pipes anyway? by jolmg on Thursday June 02, 2022 @06:19PM (#31601257)
> Historically

It's probably still common in installation scripts, like in Dockerfiles. `apt-get install` has the `-y` option, but it would be useful for all other programs that don't.

Re:How fast are Linux pipes anyway? by linsomniac on Thursday June 02, 2022 @09:30PM (#31603121)
Just to clarify: I was applying "historically" to "fsck", not to the use of "yes" in general. I can't remember the last time I've had the need to use "yes | fsck"
Re:How fast are Linux pipes anyway? by travisgriggs on Thursday June 02, 2022 @10:46AM (#31595536)
> Honest question: what are the practical use cases of this?

It also allows you to script otherwise interactive command line operations with the correct answer. Many come like tools now days provide specific options to override queries. But there are still a couple hold outs which might not.

Re:How fast are Linux pipes anyway? by TacticalCoder on Thursday June 02, 2022 @10:19AM (#31595216)
> Repeatedly typing the 'y' character into a Linux pipe is surely not that common, especially at that bit rate.

At that rate no but I definitely use it once in a while. For example if a copy quite a few files and then get repeatedly asked if I want to overwrite the destination (when it's already present). Sure, I could get my commmand back and use the proper flag to "cp" or whatever to overwrite, but it's usually much quicker to just get back the previous line, go at the beginning (C-a), then type "yes | " and be done with it.

Note that you can pass a parameter to "yes" and then it repeats what you passed instead of 'y'.

Re:How fast are Linux pipes anyway? by jolmg on Thursday June 02, 2022 @06:38PM (#31601428)
> especially at that bit rate. Also seems like the bottleneck would always be the consuming program...

It's not made to be fast; it's just fast by nature, because there's no other computation it needs to do than to just output the string.

Re:How fast are Linux pipes anyway? by kaba0 on Friday June 03, 2022 @05:56AM (#31606294)
It is optimized quite seriously. I remember there was a comparison of it with I believe a BSD version, where the latter was thousands time more readable (although slower).
Re:How fast are Linux pipes anyway? by jolmg on Friday June 03, 2022 @06:32PM (#31614515)
I'm getting ~3.10GiB/s with both GNU's and FreeBSD's. I do see that GNU's version has some optimizations, but their effectiveness isn't apparent when doing `yes | pv > /dev/null`.

However, my point was just that its performance was never a main point of it. Even without optimizations, it's still very fast, and I don't think whoever created it first was concerned with it having to be super fast, as long as it was faster than the prompts of whatever was downstream in the pipe.

Re:How fast are Linux pipes anyway? by zbentley on Saturday June 04, 2022 @07:21AM (#31619083)
It really is! It's been a few years since I saw the article on HN so I just reposted it: https://news.ycombinator.com/item?id=31619076
Re:How fast are Linux pipes anyway? by singron on Thursday June 02, 2022 @10:18AM (#31595205)
Yes can repeat any string, not just "y". It can be useful for basic load generation.
Re:How fast are Linux pipes anyway? by jolmg on Thursday June 02, 2022 @06:22PM (#31601285)
I've used it to test some db behavior with `yes 'insert ...;' | mysql ...`. Fastest insertions I could think of.