News for hackers. Stuff that matters.
Re:Benchmarking shell pipelines and the Unix “tools” philosophy by justinsaccount on Wednesday January 08, 2020 @10:41AM (#21992337)
With super small dataset sizes like that it'll fit in the L2 cache and can behave differently. I did test this though, and for 20,000 items generated with your loop:

sort | uniq -c takes .017s (fastest out of a few runs)

the awk command I used above takes .013s

A trivial implementation I have in go takes .08s

Additionally, using this 'protos' file which is 1,000,000 lines of tcp,udp,icmp:

  $ time (sort protos|uniq -c)
     5915 icmp
   332003 tcp
   662082 udp

  real 0m0.232s
  user 0m0.739s
  sys 0m0.100s

  $ # fixed to count 'lines' and not the first column, which makes it faster.
  $ time awk '{lines[$0]++} END {for (l in lines) printf("%s %d\n", l, lines[l])}'  < protos
  icmp 5915
  udp 662082
  tcp 332003

  real 0m0.194s
  user 0m0.190s
  sys 0m0.004s

  $ time ./c < protos
  662082 udp
  332003 tcp
  5915 icmp

  real 0m0.088s
  user 0m0.084s
  sys 0m0.005s
so yes, I do test my assumptions.
Re:Benchmarking shell pipelines and the Unix “tools” philosophy by dredmorbius on Wednesday January 08, 2020 @03:03PM (#21995172)
I was suggesting @crystaldev may not have.

And, more significantly, and as you've confirmed, not all in-memory processing is equivalent. There are faster and slower all-in-memory algorithms and implementations

Hrm... Maybe a bogounique implementation might be appropriate here....

(By analogy to bogosort: https://en.wikipedia.org/wiki/Bogosort)