Re:Parallel – shell tool for executing jobs in parallel usin...
by LispSporks22 on Friday July 14, 2023 @03:33PM
(#36728838)
Want to like this tool, but every time I need to use it I’ve gotta relearn the idiosyncratic syntax it uses. Lately I just use Ruby plus a couple of gems instead
[ Parent
| on HN ]
Re:Parallel – shell tool for executing jobs in parallel usin...
by stephenr on Friday July 14, 2023 @03:38PM
(#36728904)
I remember trying to work around some aspect of this tool in a shell tool for a client , and in the end abandoned that, in favour of background tasks and tracking pids, which is kind of astonishing.
I don’t actually remember the problem but possibly something to do with how output is handled.
Re:Parallel – shell tool for executing jobs in parallel usin...
by ElectricalUnion on Friday July 14, 2023 @04:08PM
(#36729259)
Almost every time I think I want this tool, I end up with a ad-hoc Python script instead, and Python is objectively speaking a terrible language for multiprocessing tasks.
Re:Parallel – shell tool for executing jobs in parallel usin...
by cempaka on Friday July 14, 2023 @05:17PM
(#36730059)
Huh, I just had the opposite experience. I wanted to parallelize the work of converting a bunch of raw camera images on my laptop to JPGs using ImageMagick, and after looking into what it would take to do it in Ruby I found that `parallel` was the better way to go.
Re:Parallel – shell tool for executing jobs in parallel usin...
by seized on Friday July 14, 2023 @11:00PM
(#36732786)
I've used it a bunch of times for some giant batch jobs across hundreds or thousands of cores... And I've found it easier to build a job file of the exact commands and redirect that into it.
Re:Parallel – shell tool for executing jobs in parallel usin...
by caro11ne on Saturday July 15, 2023 @10:24AM
(#36737048)
I use paralel often and find the syntax reasonable:
cat args | parallel this-program {}
parallel this-program {} ::: arg1 arg2 arg3 ...
The only really new thing is ::: which separates the program from the arguments if you want to give them directly.Can you show what the syntax should be for it to be intuitive to you?