News for hackers. Stuff that matters.
Re:How fast are Linux pipes anyway? by nequo on Thursday June 02, 2022 @08:51AM (#31594164)
For me:

Python3: 3 MiB/s

Node: 350 KiB/s

Lua: 12 MiB/s

  lua -e 'while true do io.write("1") end' | pv > /dev/null
Haskell: 5 MiB/s

  loop = do
    putStr "1"
    loop

  main = loop
Awk: 4.2 MiB/s

  yes | awk '{printf("1")}' | pv > /dev/null
Re:How fast are Linux pipes anyway? by VWWHFSfQ on Thursday June 02, 2022 @09:25AM (#31594567)
Lua is an interesting one.

    while true do
      io.write "1"
    end
PUC-Rio 5.1: 25 MiB/s

PUC-Rio 5.4: 25 MiB/s

LuaJIT 2.1.0-beta3: 550 MiB/s <--- WOW

They all go slightly faster if you localize the reference to `io.write`

    local write = io.write
    while true do
      write "1"
    end
Re:How fast are Linux pipes anyway? by yakubin on Thursday June 02, 2022 @10:35AM (#31595408)
> They all go slightly faster if you localize the reference to `io.write`

No noticeable difference for LuaJIT, which makes sense, since JIT should figure it out without help.

Re:How fast are Linux pipes anyway? by bjoli on Thursday June 02, 2022 @02:44PM (#31598944)
And this, folks, is why you have immutable modules. If you know before runtime what something is, lookup is a lot faster.
Re:How fast are Linux pipes anyway? by VWWHFSfQ on Thursday June 02, 2022 @11:01AM (#31595739)
Ah yes you're right. Basically no difference with LuaJIT.

5.1 and 5.4 show about ~8% improvement.

Re:How fast are Linux pipes anyway? by dllthomas on Thursday June 02, 2022 @11:36AM (#31596177)
Haskell can be even simpler:

    main = putStr (repeat '1')
[Edit: as pointed out below, this is no longer the case!]

Strings are printed one character at a time in Haskell. This choice is justified by unpredictability of the interaction between laziness and buffering; I am uncertain it's the correct choice, but the proper response is to use Text where performance is relevant.

Re:How fast are Linux pipes anyway? by nequo on Thursday June 02, 2022 @12:19PM (#31596817)
Wow, this does 160 MiB/s. That's a huge improvement! The output of strace looks completely different:

  poll([{fd=1, events=POLLOUT}], 1, 0)    = 1 ([{fd=1, revents=POLLOUT}])
  write(1, "11111111111111111111111111111111"..., 8192) = 8192
  poll([{fd=1, events=POLLOUT}], 1, 0)    = 1 ([{fd=1, revents=POLLOUT}])
  write(1, "11111111111111111111111111111111"..., 8192) = 8192
With the recursive code, it buffered the output in the same way but bugged the kernel a whole lot more in-between writes. Not exactly sure what is going on:

  poll([{fd=1, events=POLLOUT}], 1, 0)    = 1 ([{fd=1, revents=POLLOUT}])
  write(1, "11111111111111111111111111111111"..., 8192) = 8192
  rt_sigprocmask(SIG_BLOCK, [INT], [], 8) = 0
  clock_gettime(CLOCK_PROCESS_CPUTIME_ID, {tv_sec=0, tv_nsec=920390843}) = 0
  rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
  rt_sigprocmask(SIG_BLOCK, [INT], [], 8) = 0
  clock_gettime(CLOCK_PROCESS_CPUTIME_ID, {tv_sec=0, tv_nsec=920666397}) = 0
  ...
  rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
  poll([{fd=1, events=POLLOUT}], 1, 0)    = 1 ([{fd=1, revents=POLLOUT}])
  write(1, "11111111111111111111111111111111"..., 8192) = 8192
Re:How fast are Linux pipes anyway? by dllthomas on Thursday June 02, 2022 @12:28PM (#31596953)
I'm honestly surprised either of them wind up buffered! That must be a change since I stopped paying as much attention to GHC.

I'm also not sure what's going on in the second case. IIRC, at some point historically, a sufficiently tight loop could cause trouble with handling SIGINT, so it might be related to some overagressive workaround for that?