Re:Unix: When pipes get names
by smlacy on Tuesday October 08, 2013 @07:49PM
(#6518633)
Sort a file in "parallel" (note quotes!) in a multicore machine (note I'm showing 2 cores but this could be generalized to more):
It's left as an exercise for the reader to determine if this actually saves any time, and if this technique could be used to implement a truly parallel sort.
[ Parent
| on HN ]
What about?
sort --parallel=2 input_fileWell, yeah, that's the easy way. :)
This is mostly to illustrate a technique of "parallel processing using fifos and split input" It's almost like a miniature mapreduce running locally. See also "bashreduce" (https://github.com/erikfrey/bashreduce) which I didn't write.