News for hackers. Stuff that matters.
Re:Useful Unix commands for exploring data by sheetjs on Wednesday August 27, 2014 @10:23AM (#8232234)
You can do this without sorting:

    awk '!x[$0]++'
Re:Useful Unix commands for exploring data by _delirium on Wednesday August 27, 2014 @11:57AM (#8232776)
That's usually faster where possible, but it may cause problems on large data sets, since it loads the entire set of unique strings (and their counts) into an in-memory hash table.
Re:Useful Unix commands for exploring data by jingo on Wednesday August 27, 2014 @10:58AM (#8232451)
I use something like this everyday:

awk '!($0 in a);a[$0]; print}'

I rarely if ever use uniq to remove duplicates. Sorting is expensive.