The long-standing advice when writing GNU utilities used to be that if the program you were replacing was optimized for minimizing CPU use, write yours to minimize memory use, or vice-versa. Or in this case, if the program was optimized for simplicity, optimize for throughput.

It would have been very easy for the nascent GNU project to unintentionally produce a line-by-line equivalent of BSD yes.c, which would have potentially landed them in the 80/90s equivalent of the Google v.s. Oracle case.

https://www.gnu.org/prep/standards/standards.html#Reading-No...

(Lately I've been really tempted to pull a Cato and start terminating my every HN post with "C delenda est." https://en.wikipedia.org/wiki/Carthago_delenda_est)

https://github.com/coreutils/coreutils/blob/ccbd1d7dc5189f46...

https://github.com/coreutils/coreutils/commit/35217221c211f3...

But see the post of belorn where he argues that the error handling seems to be good.

This is the beautiful part of Unix: small tools that do only one thing well. Programs following this philosophy are very good abstractions. They do one very well defined thing so you can use them without having to understand how they work. I have used Unix for years and I've never felt the need to read the source code for `yes`. And because they do a very small thing, even if you need to read them, the overhead of optimization is not that much, for example, the optimized GNU yes is just under 100 LOC if you remove comments and help boilerplate. Yes, it's longer than the BSD version, but it's just a matter of minutes to understands what it does.

I'm glad it runs fast, and I hope that all OS utilities are optimised (and tested, of course!) instead of making their source code pretty. The fact is, most people want to use programs, not read them.

I want to use safe programs, and programs with readable code are more likely to be properly audited.

That really doesn't make any sense, /dev/zero should be at least as fast as yes.

I agree, all I remember is that when I tried it, /dev/zero sometimes sucked performance-wise. I can't recall the exact circumstances as it was some time ago, and could have been on any of Linux/FreeBSD/SunOS/HP-UX/IRIX - perhaps it was the fastest common way at the time?

On a recent x64 Linux, /dev/zero seems plenty fast enough now:

  $ dd bs=8k count=819200 if=/dev/zero of=/dev/null
  819200+0 records in
  819200+0 records out
  6710886400 bytes (6.7 GB, 6.2 GiB) copied, 0.331137 s, 20.3 GB/s

  $ yes | dd bs=8k count=819200 of=/dev/null
  819200+0 records in
  819200+0 records out
  6710886400 bytes (6.7 GB, 6.2 GiB) copied, 0.959551 s, 7.0 GB/s

$ pv < /dev/zero > /dev/null [ 16GiB/s]

But the version of yes using vmsplice() is even faster than that on my machine.

    pv < /usr/bin/yes > /dev/null

[Incidentally first I've heard of pv but I've known about dd for a decade or two].

    yes | pv > /dev/null

From pv's man page:

> Its standard input will be passed through to its standard output and progress will be shown on standard error.

> Maybe I just don't understand how pipes and redirection works since I rarely use Linux :(

The Windows/DOS command line has the same concepts[0], though it's probably used less often: by default a process has 3 FDs for STDIN (0), STDOUT (1) and STDERR (2).

At a shell

* you can feed a file to STDIN via <$FILE (so `</dev/zero pv` will feed the data of /dev/zero to pv), the 0 is optional

* you can pipe an output to a command (other | command) or the output of a command to an other (command | other)

* you can redirect the STDOUT or STDERR to files via 1> and 2> (the "1" is optional so 1> and > are the same thing) (you can redirect both independently)

* you can "merge" either output stream to the other by redirecting them to &FD so 1>&2 will redirect STDOUT (1) to STDERR (2) and 2>&1 will redirect STDERR (2) to STDOUT (1), you can combine that with a regular redirection so e.g. `command >foo 2>&1 ` or with a pipe (`command 2>&1 | other`)

And you can actually create more FDs to use in your pipelines[1] though I don't remember ever seeing

[0] https://support.microsoft.com/en-us/help/110930/redirecting-...

[1] http://tldp.org/LDP/abs/html/io-redirection.html

    yes | pv > /dev/null

cat /dev/zero | pv > /dev/null

uses a pipe like the 'yes' line above, but runs substantially slower on my computer than the redirect-only version.

I actually checked just now, and it looks like you'd be making twice as many system calls with /dev/zero compared to generating the data locally:

    strace dd count=1 bs=512 if=/dev/zero of=test
    open("/dev/zero", O_RDONLY)             = 3
    dup2(3, 0)                              = 0
    close(3)                                = 0
    lseek(0, 0, SEEK_CUR)                   = 0
    open("test2", O_WRONLY|O_CREAT|O_TRUNC, 0666) = 3
    dup2(3, 1)                              = 1
    close(3)                                = 0
    read(0,"\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 512) = 512
    write(1, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 512) = 512
    close(0)                                = 0
    close(1)    

The difference matters if you're benchmarking e.g. some new SSD compared to a tmpfs on a machine with 100+ GB of RAM. It's always better if the tools have less overhead, because the comparison is more meaningful.

Also consider that it can be faster to write to a local network than to disk. I've never done it, but I imagine that the kernel's not going to want to deal with your /dev/zero calls if it's spending all of its time writing to a 10GB switch. I can imagine some very specialized storage servers that could spend most of their time writing from memory buffers to a network switch, or if you're troubleshooting a slowdown in the networking itself.

    $ strace yes > /tmp/test2
    [various initialization steps]
    write(1, "y\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\n"..., 8192) = 8192
    write(1, "y\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\n"..., 8192) = 8192
    write(1, "y\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\n"..., 8192) = 8192
    write(1, "y\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\n"..., 8192) = 8192
    [...]

    $ strace -f sh -c 'yes > /tmp/test2'
    [various initialization steps]
    [pid 15839] write(1, "y\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\n"..., 8192) = 8192
    [pid 15839] write(1, "y\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\n"..., 8192) = 8192
    [pid 15839] write(1, "y\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\n"..., 8192) = 8192
    [pid 15839] write(1, "y\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\n"..., 8192) = 8192
    [...]

    open("/tmp/test2", O_WRONLY|O_CREAT|O_TRUNC, 0666) = 3 # Open the file, get FD 3
    fcntl(1, F_DUPFD, 10)                   = 10 # Copy FD 1 (the terminal, STDOUT) to FD 10 temporarily
    close(1)                                = 0  # Close original FD 1
    fcntl(10, F_SETFD, FD_CLOEXEC)          = 0  # Close FD 10 (the terminal, copy of STDOUT) when exec() is called
    dup2(3, 1)                              = 1  # Copy FD 3 (the file) to FD 1 (STDOUT)
    close(3)                                = 0  # Close FD 3 (the file's original descrptor)

    cat /dev/zero|pv >/dev/null 

but here's todays Useless Use of Cat Award http://porkmail.org/era/unix/award.html

https://news.ycombinator.com/item?id=14414610

But in that case one could probably say that Gnu awk is just not very good at input handling (as Mawk doesn't appear to benefit from an extra "cat").

  pv > /dev/null < /dev/zero

  </dev/zero pv >/dev/null

pv </dev/zero >/dev/null

which is a common way of doing it (for any command with any inputs and outputs, not just the above ones), i.e.:

command < input_source > output_dest

All three pv command invocation variants, the one here and the two above, work. And it becomes more clear why they are the same, when you know that the redirections are done by the shell + kernel, and the command (such as pv) does not even know about it. So in all three cases, it does not see the redirections, because they are already done by the time the command starts running (with its stdin and stdout redirected to / from those respective sources). And that is precisely why the command works the same whether it is reading from the keyboard or a file or pipe, and whether it is writing to the screen or a file or pipe.

[edit] That's actually inaccurate (and badly expressed), see comments below.

When I was writing raspbian images to SD cards to use on a raspabery Cat and DD took within a few seconds of each other on an operation longer then a minute. Since then I have been using cat where I could, but I didn't think to right down the numbers though.

https://news.ycombinator.com/item?id=14414610

Note that cat+gnu awk was faster than just gnu awk - but mawk was faster still (reading a not entirely small file).

And in a similar vein of gp comparing Gnu and Openbsd, note that openbsd cat is a little more convoluted than the simplest possible implementation (at least to my eyes):

https://github.com/openbsd/src/blob/master/bin/cat/cat.c

https://github.com/coreutils/coreutils/blob/master/src/cat.c

(That is, Gnu "cat" and OpenBSD "cat" are less different than Gnu "yes" and OpenBSD "yes").

    yes(){ while :; do echo "${1:-y}"; done; }

    import sys 
    while True:
      if len(sys.argv) > 1:
        print(sys.argv[1])
      else:
        print("y")

  while:; do echo "y"; done

https://github.com/openbsd/src/blob/master/usr.bin/yes/yes.c

It's essentially line for line identical to your python code...

For readability purpose it is easier to go through each lines of the python program than the OpenBSD C code. Its not massively different, but its distinguishable enough that I would choose the python version if I wanted to maximize readability, minimize syntax requirement and did not want to use shell script.

The Shell function is in my view the superior choice if the audience is a programmer than know the shell script syntax. It is just a single loop and is written in the environment that the program is intended to be used in. The only drawback there is the speed.

... and python. Don't forget to count python.

Just because you say so?

Not a huge bit of work to be sure but for a non-commercial project you might have trouble finding a volunteer who cares about that tedium.

When I saw this item here I reached for the terminal and wrote a simple (and non-compliant) alternative that simply initialises 8150 bytes of "y\n" and then loops a write forever. I understand that it is not a fully standard-compliant yes, and that maybe GNU yes is indeed fast, but that awfully simple program that takes all of 10 lines (everything included) and took me all of a minute to write performs just as well as far as pv is concerned.

(I eventually completed a feature complete yes but I still think that simply not using `puts` is hardly optimising to the limit.)

https://github.com/lattera/glibc/blob/master/string/strlen.c

https://sourceware.org/git/?p=glibc.git;a=blob;f=sysdeps/x86...

The pcmpeqb instruction is from SSE 4, it compares 16 bytes per op

Naturally it could be that the C function puts used by most people in OpenBSD is implemented with built in EINTR catching loop, or that OpenBSD do not interrupts writes.

As you say, the GNU versions definitely handle EINTR - the linux man page for puts() just says it returns a non-negative value on success, it's not even specified whether it returns the number of bytes written or not.

But the point stands; there's no error handling in the OpenBSD version. But that could be considered a design decision; the OpenBSD version never gives up on writing data until you kill it; the GNU version bails at the first error.

I think unix v6 was mostly as clear. Linux suffered the real world penalty. But maybe BSD managed to keep it's source poetic.

atexit (close_stdout);

Aren't all streams closed at exit? And why closing stdout, anyway?

https://git.busybox.net/busybox/tree/coreutils/yes.c

Why would it be slower to do a single, say, 8190 bytes write instead of 2730 3-byte writes?

Small write calls themselves incur a considerable syscall overhead.

It's just a trade off. For utilities whose behaviour doesn't change we're happy to improve the speed.

In the old days I think we would have left yes in c because the compiler will build it on any platform.

My first experience with Linux is porting land.c to one of the commercial Unix. Now I first met TCP/IP on a 3B2 and later met systems sold by SMI and DEC and SCO and we all mostly constructed packets the same way and a guy called Stevens had written some nice books about this that everyone had. I think I recall the commercial and free BSD also did things the usual way. But whoever figured out this interesting phenomenon land.c demonstrated happened to be a Linux user and this platform had some different ideas about it. I saw rewriting the relevant parts as an annoying, menial task.