News for hackers. Stuff that matters.
Re:Benchmarking shell pipelines and the Unix “tools” philosophy by justinsaccount on Tuesday January 07, 2020 @09:00PM (#21987502)
Are you being difficult on purpose?

I posted a comment on how 'sort | uniq -c | sort -n' is an interesting and very capable pipeline, but often misused and slower than other alternatives.

> you are comparing

Yes, I am comparing two methods of accomplishing the same thing. That is how comparing things works.

> Please, "huge waste"? How do you sort something that does not fit in memory?

Note how the full sentence included "if you give it 100GB of 5 different strings". If your input is 100GB of 5 different strings, then the hash table will easily fit in memory, and sorting the entire data set only to pass it to 'uniq -c' is indeed a 'huge waste'.

There are tons of large data sets that only have a small number of unique values in particular fields. protocols, ports, http status codes, hour of the day, etc. 'sort | uniq -c | sort -n' will work for all of them, but not nearly as efficient a hash table.

Re:Benchmarking shell pipelines and the Unix “tools” philosophy by tuldia on Wednesday January 08, 2020 @02:50AM (#21989275)
> Are you being difficult on purpose?

Programming is about paying the bare minimum attention to the details.

> [...] two methods of accomplishing the same thing [...]

Absolutelly not.

one prints:

  72000000 hello
  72000000 world
the other

  hello 72000000
  world 72000000
Now try both examples against a file with more than one column to understand what I'm talking about ;)