Re:Linux Productivity Tools (2019) [pdf]
by earthscienceman on Monday May 18, 2020 @07:10PM
(#23229460)
Wow. Well, I wish I would have known abut GNU parallel before I wrote that massive bash script a few years ago. Thanks for posting this.
[ Parent
| on HN ]
Re:Linux Productivity Tools (2019) [pdf]
by yjftsjthsd-h on Monday May 18, 2020 @07:36PM
(#23229650)
Learning more unix tools tends to lead to that feeling a lot. The last few decades of devs building tools generally means that most obvious problems have a solution, but of course one usually only learns about that solution after hacking together their own version:) One tip: Look up moreutils; it's a nice consolidation of things that make a lot of things easier. (One gotcha: moreutils ships its own different parallel tool.)
Its a very handy utility. Sometimes when I am lazy to write proper multiprocessing support in my codes, I get some work started anyway by (1) quickly providing a basic CLI version of my code (2) throwing in a bunch of code-invocation commands in a file and calling GNU parallel on it.
xargs can do most of the common tasks perl parallel does, see xars -P
Imho, the main advantage of Parallel is that it can execute the tasks on multiple hosts, straight from the man page:
> GNU parallel is a shell tool for executing jobs in parallel using one or more computers. The typical input is a list of files, a list of hosts, a list of users, a list of URLs, or a list of tables.
Re:Linux Productivity Tools (2019) [pdf]
by mechanical_jane on Wednesday May 20, 2020 @08:17AM
(#23245954)
This gives an (artificial) example of the power of GNU Parallel over xargs: https://unix.stackexchange.com/questions/405552/using-xargs-...
It's an absolute godsend for plowing through data-parallel computations.
Re:Linux Productivity Tools (2019) [pdf]
by arminiusreturns on Monday May 18, 2020 @10:36PM
(#23230781)
It's great for scaling text smashing and bashing imho.