News for hackers. Stuff that matters.
Re:Ask HN: Sorting massive text files? by aw3c2 on Saturday August 14, 2010 @12:03PM (#1603465)
I guess a line count has to run through the whole file, so obviously your storage is the bottleneck. Let's say your storage manages 100MB/s constantly for the whole 35GB it would take about 6 minutes.

Your 'cat' is not needed by the way, sort takes a filename as argument. And sort can '-u'!

It would be interesting if sort would fail on this (why?) or how long it would take.

edit: Why on earth are you doing this as root?

There are no comments on this comment yet.