Re:Cheating at a company group activity using Unix tools
by dagw on Monday December 13, 2021 @09:25AM
(#29539819)
xargs automatically parallelizes programs. How can this be done efficiently (meaning I don't have to write much additional code) in your proposed "real language"?
By using xargs :)
Or more likely gnu parallel. But seriously, in so many cases using GNU parallel to parallelize a process is the quickest and easiest way to approach the problem, and I use it all the time. If I need to process 10k+ images in a folder, rather than try to parallelize the process in my python/C script I'll write the fastest possible single threaded script that takes its input args as command line arguments and then uses gnu parallel to distribute the workload. The added advantage is that I can distribute this work on a cluster of machines with only a few changes to GNU Parallel's command line arguments.
By using xargs :)
Or more likely gnu parallel. But seriously, in so many cases using GNU parallel to parallelize a process is the quickest and easiest way to approach the problem, and I use it all the time. If I need to process 10k+ images in a folder, rather than try to parallelize the process in my python/C script I'll write the fastest possible single threaded script that takes its input args as command line arguments and then uses gnu parallel to distribute the workload. The added advantage is that I can distribute this work on a cluster of machines with only a few changes to GNU Parallel's command line arguments.