News for hackers. Stuff that matters.
Re:How is GNU `yes` so fast? by joosters on Tuesday June 13, 2017 @05:48AM (#14543757)
I've used 'yes' many times to generate huge amounts of data quickly. Back then, it never had the small string optimisation, but you could always run 'yes InsertReallyLongStringHere' to spew out data much faster than /dev/urandom or even /dev/zero

I'm glad it runs fast, and I hope that all OS utilities are optimised (and tested, of course!) instead of making their source code pretty. The fact is, most people want to use programs, not read them.

Re:How is GNU `yes` so fast? by currysausage on Tuesday June 13, 2017 @09:18AM (#14544475)
> The fact is, most people want to use programs, not read them.

I want to use safe programs, and programs with readable code are more likely to be properly audited.

Re:How is GNU `yes` so fast? by joosters on Tuesday June 13, 2017 @09:23AM (#14544503)
Audited UNIX tools - does such a thing exist?
Re:How is GNU `yes` so fast? by _cx2w on Tuesday June 13, 2017 @09:28AM (#14544540)
openbsd.org
Re:How is GNU `yes` so fast? by khedoros1 on Tuesday June 13, 2017 @01:45PM (#14546832)
Sounds like you want to stick to something with a non-GNU userspace, apparently.
Re:How is GNU `yes` so fast? by masklinn on Tuesday June 13, 2017 @07:06AM (#14543965)
> you could always run 'yes InsertReallyLongStringHere' to spew out data much faster than /dev/zero

That really doesn't make any sense, /dev/zero should be at least as fast as yes.

Re:How is GNU `yes` so fast? by joosters on Tuesday June 13, 2017 @07:19AM (#14544002)
/dev/zero should be at least as fast as yes

I agree, all I remember is that when I tried it, /dev/zero sometimes sucked performance-wise. I can't recall the exact circumstances as it was some time ago, and could have been on any of Linux/FreeBSD/SunOS/HP-UX/IRIX - perhaps it was the fastest common way at the time?

On a recent x64 Linux, /dev/zero seems plenty fast enough now:

  $ dd bs=8k count=819200 if=/dev/zero of=/dev/null
  819200+0 records in
  819200+0 records out
  6710886400 bytes (6.7 GB, 6.2 GiB) copied, 0.331137 s, 20.3 GB/s

  $ yes | dd bs=8k count=819200 of=/dev/null
  819200+0 records in
  819200+0 records out
  6710886400 bytes (6.7 GB, 6.2 GiB) copied, 0.959551 s, 7.0 GB/s
Re:How is GNU `yes` so fast? by Tepix on Tuesday June 13, 2017 @09:08AM (#14544425)
No need for "dd", let "pv" get the data from /dev/zero directly:

$ pv < /dev/zero > /dev/null [ 16GiB/s]

But the version of yes using vmsplice() is even faster than that on my machine.

Re:How is GNU `yes` so fast? by pbhjpbhj on Tuesday June 13, 2017 @09:55AM (#14544686)
What's the line to test `yes` with `pv`?

    pv < /usr/bin/yes > /dev/null
doesn't seem to work properly. FWIW I get 330MiB/s vs 8.4GiB/s for /dev/zero.

[Incidentally first I've heard of pv but I've known about dd for a decade or two].

Re:How is GNU `yes` so fast? by masklinn on Tuesday June 13, 2017 @11:37AM (#14545579)
< and > are for file redirection, yes is a binary so you want to pipe its stdout into pv:

    yes | pv > /dev/null
Re:How is GNU `yes` so fast? by pbhjpbhj on Wednesday June 14, 2017 @09:52AM (#14552678)
So how did that redirect even work; should we be doing a `mknod` to make a "yes" device to make the comparison work (can we, does it help other than in my naive imagination).
Re:How is GNU `yes` so fast? by BenjiWiebe on Wednesday June 14, 2017 @02:26PM (#14554824)
The reason that redirect worked is because it was using the contents of the yes program instead of its output.
Re:How is GNU `yes` so fast? by kevindqc on Tuesday June 13, 2017 @03:39PM (#14547911)
I know this works, but how come we can see the output of pv when it is redirected to /dev/null? Maybe I just don't understand how pipes and redirection works since I rarely use Linux :(
Re:How is GNU `yes` so fast? by masklinn on Wednesday June 14, 2017 @03:57AM (#14551286)
> I know this works, but how come we can see the output of pv when it is redirected to /dev/null?

From pv's man page:

> Its standard input will be passed through to its standard output and progress will be shown on standard error.

> Maybe I just don't understand how pipes and redirection works since I rarely use Linux :(

The Windows/DOS command line has the same concepts[0], though it's probably used less often: by default a process has 3 FDs for STDIN (0), STDOUT (1) and STDERR (2).

At a shell

* you can feed a file to STDIN via <$FILE (so `</dev/zero pv` will feed the data of /dev/zero to pv), the 0 is optional

* you can pipe an output to a command (other | command) or the output of a command to an other (command | other)

* you can redirect the STDOUT or STDERR to files via 1> and 2> (the "1" is optional so 1> and > are the same thing) (you can redirect both independently)

* you can "merge" either output stream to the other by redirecting them to &FD so 1>&2 will redirect STDOUT (1) to STDERR (2) and 2>&1 will redirect STDERR (2) to STDOUT (1), you can combine that with a regular redirection so e.g. `command >foo 2>&1 ` or with a pipe (`command 2>&1 | other`)

And you can actually create more FDs to use in your pipelines[1] though I don't remember ever seeing

[0] https://support.microsoft.com/en-us/help/110930/redirecting-...

[1] http://tldp.org/LDP/abs/html/io-redirection.html

Re:How is GNU `yes` so fast? by wutbrodo on Tuesday June 13, 2017 @03:54PM (#14548043)
You're redirecting stdout to /dev/null and pv is writing to stderr. If you use &> instead, stderr and stdout will both be redirected to /dev/null and you will see no output at all.
Re:How is GNU `yes` so fast? by moosingin3space on Tuesday June 13, 2017 @10:13AM (#14544805)
Maybe this?

    yes | pv > /dev/null
Re:How is GNU `yes` so fast? by wutbrodo on Tuesday June 13, 2017 @03:56PM (#14548064)
That's not quite apples-to-apples though:

cat /dev/zero | pv > /dev/null

uses a pipe like the 'yes' line above, but runs substantially slower on my computer than the redirect-only version.