News for hackers. Stuff that matters.
Re:Parallel – shell tool for executing jobs in parallel usin... by LispSporks22 on Friday July 14, 2023 @03:33PM (#36728838)
Want to like this tool, but every time I need to use it I’ve gotta relearn the idiosyncratic syntax it uses. Lately I just use Ruby plus a couple of gems instead
[ Parent | on HN ]
Re:Parallel – shell tool for executing jobs in parallel usin... by stephenr on Friday July 14, 2023 @03:38PM (#36728904)
I remember trying to work around some aspect of this tool in a shell tool for a client , and in the end abandoned that, in favour of background tasks and tracking pids, which is kind of astonishing.

I don’t actually remember the problem but possibly something to do with how output is handled.

[ Parent | on HN ]
Re:Parallel – shell tool for executing jobs in parallel usin... by ElectricalUnion on Friday July 14, 2023 @04:08PM (#36729259)
Almost every time I think I want this tool, I end up with a ad-hoc Python script instead, and Python is objectively speaking a terrible language for multiprocessing tasks.
[ Parent | on HN ]
Re:Parallel – shell tool for executing jobs in parallel usin... by cempaka on Friday July 14, 2023 @05:17PM (#36730059)
Huh, I just had the opposite experience. I wanted to parallelize the work of converting a bunch of raw camera images on my laptop to JPGs using ImageMagick, and after looking into what it would take to do it in Ruby I found that `parallel` was the better way to go.
[ Parent | on HN ]
Re:Parallel – shell tool for executing jobs in parallel usin... by seized on Friday July 14, 2023 @11:00PM (#36732786)
I've used it a bunch of times for some giant batch jobs across hundreds or thousands of cores... And I've found it easier to build a job file of the exact commands and redirect that into it.
[ Parent | on HN ]
Re:Parallel – shell tool for executing jobs in parallel usin... by caro11ne on Saturday July 15, 2023 @10:24AM (#36737048)
I use paralel often and find the syntax reasonable:

    cat args | parallel this-program {}
    parallel this-program {} ::: arg1 arg2 arg3 ...
The only really new thing is ::: which separates the program from the arguments if you want to give them directly.

Can you show what the syntax should be for it to be intuitive to you?

[ Parent | on HN ]
Re:Parallel – shell tool for executing jobs in parallel usin... by stabbles on Friday July 14, 2023 @03:43PM (#36728958)
GNU parallel devs really want you to cite their work... it's just a tool folks, it's like citing the manual of a hammer.
[ Parent | on HN ]
Re:Parallel – shell tool for executing jobs in parallel usin... by musicale on Friday July 14, 2023 @05:14PM (#36730009) 

    #!/usr/bin/python
    print(
    """Acknowledgments: This Python[1] script runs on Linux[2] based operating systems
    and uses the ls (see list-segments[3]) utility, written in the ANSI C[4] language.

    [1] Python programming language, Van Rossum et al., 1991-
    [2] Linux kernel, Torvalds et al., 1991-
    [3] Multics operating system, Corbató et al., 1969-
    [4] C Programming Language 2nd. ed., Kernighan, Ritchie, 1988""")
    ...
[ Parent | on HN ]
Re:Parallel – shell tool for executing jobs in parallel usin... by peacebreaker2k on Friday July 14, 2023 @05:19PM (#36730089)
it was so good, it inspired me to put this out there - https://github.com/korovkin/parallel
[ Parent | on HN ]
Re:Parallel – shell tool for executing jobs in parallel usin... by theamk on Friday July 14, 2023 @05:38PM (#36730274)
* * *
[ Parent | on HN ]