News for hackers. Stuff that matters.

A comment on An overengineered solution to `sort | uniq -c` with 25x throughput (hist). Parent comment

Re:An overengineered solution to `sort | uniq -c` with 25x t... by zahlman on Monday October 27, 2025 @10:42AM (#45721528)
How often is "count the unique lines of a file" a realistic task for others out there, and how big of files do y'all need to process and why?
[ Parent | on HN ]
Re:An overengineered solution to `sort | uniq -c` with 25x t... by Tostino on Monday October 27, 2025 @11:05AM (#45721822)
Reasonably often in ETL type tasks.
[ Parent | on HN ]
Re:An overengineered solution to `sort | uniq -c` with 25x t... by noamteyssier on Tuesday October 28, 2025 @12:52PM (#45735355)
Shows up a lot in bioinformatics actually - trying to identify sequences with a specific subsequence (grep) and how many of each unique sequence there are. The number of lines here could be massive (order of 1-10's of GB)

You don't really end up using these results in any specific analysis but it's super helpful for troubleshooting tools or edge-cases.

[ Parent | on HN ]