News for hackers. Stuff that matters.
Re:Unix command line queue utility by justin_vanw on Wednesday August 26, 2015 @10:21PM (#10127179)

   $ mkfifo ~/.jobqueue
   $ parallel :::: ~/.jobqueue
now, from another terminal:

   echo ls > ~/.jobqueue
Tada!

for a remote job queue, use nc instead of a fifo

for a job queue with a buffer, use a file, and run $ parallel :::: <(tail -f thefile) you can even truncate the file from time to time.

Of course, this doesn't capture the context you are in, such as your current directory or the value of shell variables.

Re:Unix command line queue utility by kovek on Thursday August 27, 2015 @01:50AM (#10127743)
Here is an alternative:

  tty1$ mkfifo /tmp/jobqueue
  tty1$ eval $(tail -n 1 /tmp/jobqueue)
And then, in another terminal:

  tty2$ echo ls > /tmp/jobqueue
Re:Unix command line queue utility by giis on Thursday August 27, 2015 @12:33AM (#10127558)
What's 'parallel'? Is it a *nix command? Looks like its not available on Fedora box
Re:Unix command line queue utility by teraflop on Thursday August 27, 2015 @12:52AM (#10127608)
Yes, it's a GNU tool. Try "yum install parallel".

http://www.gnu.org/software/parallel/

Re:Unix command line queue utility by kovek on Wednesday August 26, 2015 @07:35PM (#10126633)
This stackoverflow answer[0] shows how you can queue jobs even after you start running other processes in your queue. Essentially, you can do:

  sleep 2; echo "A" && sleep 2 ; echo "B" && sleep 2 ; echo "C" 
  # Quickly Ctrl-Z
  fg && sleep 2 ; echo "D"
and you will see A\nB\nC\nD\n slowly printed out in your terminal.

[0]: http://superuser.com/a/345455

Re:Unix command line queue utility by CyberShadow on Wednesday August 26, 2015 @05:44PM (#10126218)
Could you please add some usage examples or elaborate on when this tool would be useful? For example,

> building several targets of a Makefile

As opposed to `make && sudo make install` ?

> downloading multiple files one at a time

As opposed to `(wget url1 &) ; (wget url2 &)` ?

> or simply as a glorified `nohup`

As opposed to `nohup` ?

Re:Unix command line queue utility by cozzyd on Wednesday August 26, 2015 @06:18PM (#10126369)
I imagine it's useful if you don't know all the commands you want to queue at the beginning.

For example:

    nq ./my_awesome_analysis /data/complicated_file_with_long_name.dat

    # um... what's the next file? 

    ls /data 
    nq ./my_awesome_analysis /data/even_more_complicated_path.dat

    man rsync
    nq rsync -avh --progress out* somewhere_else.net:
Re:Unix command line queue utility by jessaustin on Wednesday August 26, 2015 @06:32PM (#10126428)
You don't need nq for that. Just kick off ./my_lame_analysis in the background whenever you want.
Re:Unix command line queue utility by cozzyd on Wednesday August 26, 2015 @06:59PM (#10126505)
Yes but then they will execute at the same time instead of sequentially.
Re:Unix command line queue utility by shanemhansen on Wednesday August 26, 2015 @07:20PM (#10126583)
How about:

    ./my_awesome_analysis /data/complicated_file_with_long_name.dat &
    # do some stuf
    wait
    ./my_awesome_analysis /data/even_more_complicated_path.dat
Re:Unix command line queue utility by e40 on Wednesday August 26, 2015 @07:33PM (#10126626)
It's still not equivalent, and it's more cumbersome.
Re:Unix command line queue utility by PeterWhittaker on Wednesday August 26, 2015 @05:51PM (#10126248)
Thank you for making the comment I came to make!

The shells have had queueing for, well, a very, very long time....

Re:Unix command line queue utility by jschwartzi on Wednesday August 26, 2015 @05:37PM (#10126188)
Which systems guarantee monotonic behavior for gettimeofday()? I can't think of any. I hope there's no behavior in nq which relies on monotonicity other than file naming, because otherwise you might run into some obscure issues.
Re:Unix command line queue utility by pwg on Wednesday August 26, 2015 @06:22PM (#10126385)
How does this differ from task spooler:

http://vicerveza.homeunix.net/~viric/soft/ts/

Re:Unix command line queue utility by dsl on Wednesday August 26, 2015 @07:04PM (#10126527)
cat list.txt | xargs -n 1 -P 20 wget

Queues up 20 concurrent downloads with wget, for example.

Re:Unix command line queue utility by icebraining on Wednesday August 26, 2015 @07:17PM (#10126576)
You could also use a for loop. But as others pointed out, that doesn't allow you to add items to the queue after it has started.
Re:Unix command line queue utility by stonogo on Wednesday August 26, 2015 @10:54PM (#10127271)
It does if you use a fifo instead of a text file. This is the difference between basic Unix literacy and actual Unix proficiency. I'm not being insulting; very few people take the time to learn the tools Unix provides. And naturally, a developer is going to be more interested in writing tools than most.
Re:Unix command line queue utility by icebraining on Thursday August 27, 2015 @07:27AM (#10128742)
I don't think you can use a fifo with xargs, though it'd definitively work with a loop. Even with text files, using tail -f.
Re:Unix command line queue utility by iokanuon on Thursday August 27, 2015 @07:36AM (#10128779)

    tail -n+0 -f list.txt | xargs -n 1 -P 20 wget