News for hackers. Stuff that matters.
Re:How is GNU `yes` so fast? by masklinn on Tuesday June 13, 2017 @07:06AM (#14543965)
> you could always run 'yes InsertReallyLongStringHere' to spew out data much faster than /dev/zero

That really doesn't make any sense, /dev/zero should be at least as fast as yes.

Re:How is GNU `yes` so fast? by joosters on Tuesday June 13, 2017 @07:19AM (#14544002)
/dev/zero should be at least as fast as yes

I agree, all I remember is that when I tried it, /dev/zero sometimes sucked performance-wise. I can't recall the exact circumstances as it was some time ago, and could have been on any of Linux/FreeBSD/SunOS/HP-UX/IRIX - perhaps it was the fastest common way at the time?

On a recent x64 Linux, /dev/zero seems plenty fast enough now:

  $ dd bs=8k count=819200 if=/dev/zero of=/dev/null
  819200+0 records in
  819200+0 records out
  6710886400 bytes (6.7 GB, 6.2 GiB) copied, 0.331137 s, 20.3 GB/s

  $ yes | dd bs=8k count=819200 of=/dev/null
  819200+0 records in
  819200+0 records out
  6710886400 bytes (6.7 GB, 6.2 GiB) copied, 0.959551 s, 7.0 GB/s
Re:How is GNU `yes` so fast? by Tepix on Tuesday June 13, 2017 @09:08AM (#14544425)
No need for "dd", let "pv" get the data from /dev/zero directly:

$ pv < /dev/zero > /dev/null [ 16GiB/s]

But the version of yes using vmsplice() is even faster than that on my machine.

Re:How is GNU `yes` so fast? by pbhjpbhj on Tuesday June 13, 2017 @09:55AM (#14544686)
What's the line to test `yes` with `pv`?

    pv < /usr/bin/yes > /dev/null
doesn't seem to work properly. FWIW I get 330MiB/s vs 8.4GiB/s for /dev/zero.

[Incidentally first I've heard of pv but I've known about dd for a decade or two].

Re:How is GNU `yes` so fast? by masklinn on Tuesday June 13, 2017 @11:37AM (#14545579)
< and > are for file redirection, yes is a binary so you want to pipe its stdout into pv:

    yes | pv > /dev/null
Re:How is GNU `yes` so fast? by pbhjpbhj on Wednesday June 14, 2017 @09:52AM (#14552678)
So how did that redirect even work; should we be doing a `mknod` to make a "yes" device to make the comparison work (can we, does it help other than in my naive imagination).
Re:How is GNU `yes` so fast? by BenjiWiebe on Wednesday June 14, 2017 @02:26PM (#14554824)
The reason that redirect worked is because it was using the contents of the yes program instead of its output.
Re:How is GNU `yes` so fast? by kevindqc on Tuesday June 13, 2017 @03:39PM (#14547911)
I know this works, but how come we can see the output of pv when it is redirected to /dev/null? Maybe I just don't understand how pipes and redirection works since I rarely use Linux :(
Re:How is GNU `yes` so fast? by masklinn on Wednesday June 14, 2017 @03:57AM (#14551286)
> I know this works, but how come we can see the output of pv when it is redirected to /dev/null?

From pv's man page:

> Its standard input will be passed through to its standard output and progress will be shown on standard error.

> Maybe I just don't understand how pipes and redirection works since I rarely use Linux :(

The Windows/DOS command line has the same concepts[0], though it's probably used less often: by default a process has 3 FDs for STDIN (0), STDOUT (1) and STDERR (2).

At a shell

* you can feed a file to STDIN via <$FILE (so `</dev/zero pv` will feed the data of /dev/zero to pv), the 0 is optional

* you can pipe an output to a command (other | command) or the output of a command to an other (command | other)

* you can redirect the STDOUT or STDERR to files via 1> and 2> (the "1" is optional so 1> and > are the same thing) (you can redirect both independently)

* you can "merge" either output stream to the other by redirecting them to &FD so 1>&2 will redirect STDOUT (1) to STDERR (2) and 2>&1 will redirect STDERR (2) to STDOUT (1), you can combine that with a regular redirection so e.g. `command >foo 2>&1 ` or with a pipe (`command 2>&1 | other`)

And you can actually create more FDs to use in your pipelines[1] though I don't remember ever seeing

[0] https://support.microsoft.com/en-us/help/110930/redirecting-...

[1] http://tldp.org/LDP/abs/html/io-redirection.html

Re:How is GNU `yes` so fast? by wutbrodo on Tuesday June 13, 2017 @03:54PM (#14548043)
You're redirecting stdout to /dev/null and pv is writing to stderr. If you use &> instead, stderr and stdout will both be redirected to /dev/null and you will see no output at all.
Re:How is GNU `yes` so fast? by moosingin3space on Tuesday June 13, 2017 @10:13AM (#14544805)
Maybe this?

    yes | pv > /dev/null
Re:How is GNU `yes` so fast? by wutbrodo on Tuesday June 13, 2017 @03:56PM (#14548064)
That's not quite apples-to-apples though:

cat /dev/zero | pv > /dev/null

uses a pipe like the 'yes' line above, but runs substantially slower on my computer than the redirect-only version.