Shell Magic: Set Operations with uniq

Re:Shell Magic: Set Operations with uniq by alpb on Tuesday May 29, 2018 @08:24PM (#17183879)

Indeed comm will work much faster than uniq in this case, because it reads each file only once, and does a single pass on each file.

What the blog post does (doing cat multiple times, and then counting the occurrences) is going to be much slower, although mathematically correct.

[ Parent | on HN ]

Re:Shell Magic: Set Operations with uniq by fiddlerwoaroof on Wednesday May 30, 2018 @12:27AM (#17185064)

Doesn't uniq only read the input once? I've always assumed that the reason uniq assumes its input is presorted is that, then, it doesn't need to buffer all of it: instead, it just eliminates successive duplicate lines.

[ Parent | on HN ]

Re:Shell Magic: Set Operations with uniq by alpb on Wednesday May 30, 2018 @12:30PM (#17189103)

Yeah but in this case the author is doing "cat 1.txt 1.txt 2.txt".

[ Parent | on HN ]