Re:Benchmarking shell pipelines and the Unix “tools” philosophy
by dredmorbius on Wednesday January 08, 2020 @03:03PM
(#21995172)
I was suggesting @crystaldev may not have.
And, more significantly, and as you've confirmed, not all in-memory processing is equivalent. There are faster and slower all-in-memory algorithms and implementations
Hrm... Maybe a bogounique implementation might be appropriate here....
(By analogy to bogosort: https://en.wikipedia.org/wiki/Bogosort)
sort | uniq -c takes .017s (fastest out of a few runs)
the awk command I used above takes .013s
A trivial implementation I have in go takes .08s
Additionally, using this 'protos' file which is 1,000,000 lines of tcp,udp,icmp:
so yes, I do test my assumptions.