Re:Show HN: Radix sort big files in memory
by antender on Sunday May 10, 2020 @12:01AM
(#23130188)
After consulting with GNU sort manual:
sort has -m option just for the case of merging presorted files, so you can test this by using 'split -l', then 'xargs sort' (to parallelize), then 'sort -m' to merge chunks
[ Parent
| on HN ]
I agree that a significant proportion of time is spent on IO. Only 8m38s is actually spent sorting (out of 19m37s). However in the past my experiments have shown that using `sort -m` to sort chunks is much much slower than using `sort -S100%`.