An overengineered solution to `sort | uniq -c` with 25x throughput (hist)

Re:An overengineered solution to `sort | uniq -c` with 25x t... by Someone on Monday October 27, 2025 @03:42AM (#45718318)

> I am measuring the performance of equivalent cat <file> | sort | uniq -c | sort -n functionality.

It likely won’t matter much here, but invoking cat is unnecessary.

   sort <file> | uniq -c | sort -n

will do the job just fine. GNU’s sort also has a few flags controlling buffer size and parallelism. Those may matter more (see https://www.gnu.org/software/coreutils/manual/html_node/sort...)

[ Parent | on HN ]

Re:An overengineered solution to `sort | uniq -c` with 25x t... by kees99 on Monday October 27, 2025 @03:42PM (#45725405)

GNU coreutils version of cat doesn't do splice(), so it does slow things a tiny bit.

[ Parent | on HN ]

Re:An overengineered solution to `sort | uniq -c` with 25x t... by noamteyssier on Tuesday October 28, 2025 @12:44PM (#45735235)

Thanks for sharing!

You're right that the `cat` is unnecessary - and removing it actually had some marginal gains to the naive solution. I've updated the benchmarks to show this

Cheers

[ Parent | on HN ]