Re:Show HN: Radix sort big files in memory
by jepcommenter on Saturday May 09, 2020 @12:16PM
(#23125985)
Full line comparison would probably use memcmp that bails out on first non matching character while field-splitting overhead might be significant.
Re:Show HN: Radix sort big files in memory
by afiodorov on Saturday May 09, 2020 @04:15PM
(#23127942)
Exact same file has been taking more than 35 minutes to sort already, so it's slower without splitting.
Edit: it finished!
real 35m28.370s
user 40m17.129s
sys 4m31.081s
Where did you find the dataset, or did you construct your own?
$ stat --printf="%s\n" p.csv
1258291200
$ time sort -t, -k1 -S100% -o sorted.csv p.csv
real 0m50,186s user 4m6,962s sys 0m4,562s
$ time sort -o sorted.csv p.csv
real 0m43,483s user 3m36,473s sys 0m4,282s