Re:A small primer on xargs
by DHowett on Monday May 14, 2012 @02:46AM
(#3969792)
"In particular, the presence of \0 in your data will throw xargs for a loop (no pun intended). In this case, it is necessary to use the -0 option. This is particularly relevant if you are using the find command and supplying the -print0 option."
This glosses over the fact that those two things are intended to be used together. Delimiting filenames with a \0 is superior to delimiting them with spaces or newlines, as filenames cannot usually contain ASCII NUL; xargs and find can therefore be used to remove heterogeneous lists of files.
Additionally, the example given may better be worded as "find . -name '* junk* -exec rm '{}' \;" or even "find . -name '* junk* | xargs rm". (as given it strikes me as a useless use of grep and is additionally fragile if ls is aliased to "ls --color=always", spewing ANSI into
EDIT: Please excuse the additional spaces in the find command arguments, they are a formatting avoidance feature.
Re:A small primer on xargs
by darklajid on Monday May 14, 2012 @04:04AM
(#3969993)
If we're ripping this sample apart, I'd like to point out that
rm *junk*
in this case would've done the same, without spawning lots of useless stuff.
xargs is cool. The example fails to show a decent use case. For me a line like that would indicate that someone is _not_ familiar with his tools. Similar to
cat somefile | grep something
instead of
grep something somefile
Pipes and processes aren't endangered or (in practice) limited, but we still don't need to be wasteful?
Re:A small primer on xargs
by babarock on Monday May 14, 2012 @04:41AM
(#3970057)
Furthermore, parsing the output of `ls` is a terrible practice[1] that makes for an even poorer sample.
A little further in the article, the author even suggests running
`ls | grep junk`
(instead of the much simpler `ls junk*`. Do I need to point that out?).
I also know some people are going to retort that wasting a process or a pipe is not that bad. This discussion has been had over and over again, probably ever since Unix was invented.
In any case, the poor example chosen, as well as the flaky commands used, seriously hinder the credibility of the article.
Re:A small primer on xargs
by asdfaoeu on Monday May 14, 2012 @05:34AM
(#3970208)
If you still want to use xargs and ls perhaps avoiding \0's (perhaps so you can easily grep on the data) you can use "ls -Q | xargs rm". The advantage of that is you can still easily use grep on the data as it is still newline delimited but you won't have problems with spaces.
Also ls is never aliased to "ls --color=always" unless you've done that more likely is its aliased to "ls --color=auto" or similar which will work normally.
Re:A small primer on xargs
by robertskmiles on Monday May 14, 2012 @05:51AM
(#3970254)
> more likely is its aliased to "ls --color=auto" or similar which will work normally.
My ls (which I haven't much reconfigured) behaves very differently depending on whether it's outputting to a shell or a pipe or redirect. Not just colours, but columning and filetype indicators as well.
Re:A small primer on xargs
by gizzlon on Monday May 14, 2012 @03:35AM
(#3969928)
"What xargs is doing is taking each line of input and applying the supplied argument or command to the line of input"
Actually it does not run command once for each line of input. This confused me for some time until I read the man page:
"The command line for command is built up until it reaches a system-defined limit (unless the -n and -L options are used). The specified command will be invoked as many times as necessary to use up the list of input items. In general, there will be many fewer invocations of command than there were items in the input. This will normally have significant performance benefits. Some commands can usefully be executed in parallel too; see the -P option"
Think you're looking for "-n 1" if you want that behavior..
Re:A small primer on xargs
by slashclee on Monday May 14, 2012 @03:27AM
(#3969900)
xargs is so 20th-century. parallel[0] is way more awesome (especially with all the multi-core systems everybody has now). Cool bonus: all the things you learn for xargs work in parallel too!
Re:A small primer on xargs
by knewter on Monday May 14, 2012 @09:04AM
(#3970845)
A billion times, this. I just learned of GNU Parallel about 3 months ago and wish everyone knew about it. Have seen it used to stunning effect with a comically large pipeline (OCR patents) across a comically large number of EC2 instances. Very awesome indeed.
Re:A small primer on xargs
by webreac on Monday May 14, 2012 @03:22AM
(#3969884)
I try to use perl instead of sed, awk, bash each time the needs are beyond basic usage. This applies also sometimes for xargs for example with the following command.
find . -name junk\* | perl -nle unlink
My favorite usage of xargs if for parallel operations:
cat hostlist | xargs -P 0 -n 1 -I host rsh host gzip -9 log/*.log
Re:A small primer on xargs
by bitops on Monday May 14, 2012 @10:19AM
(#3971154)
OP here - thanks all for the comments - so yes, it looks like I don't know xargs as well as I thought I did. I'll update the post to be more accurate.
As for the points about a bad example, I am fully aware that it is extremely contrived. The point was to not confuse anyone with commands they don't already know. I'm wary of posting examples that necessitate people learning 4-5 new commands when their focus is learning just one.
This glosses over the fact that those two things are intended to be used together. Delimiting filenames with a \0 is superior to delimiting them with spaces or newlines, as filenames cannot usually contain ASCII NUL; xargs and find can therefore be used to remove heterogeneous lists of files.
Additionally, the example given may better be worded as "find . -name '* junk* -exec rm '{}' \;" or even "find . -name '* junk* | xargs rm". (as given it strikes me as a useless use of grep and is additionally fragile if ls is aliased to "ls --color=always", spewing ANSI into
EDIT: Please excuse the additional spaces in the find command arguments, they are a formatting avoidance feature.
xargs is cool. The example fails to show a decent use case. For me a line like that would indicate that someone is _not_ familiar with his tools. Similar to
instead of Pipes and processes aren't endangered or (in practice) limited, but we still don't need to be wasteful?A little further in the article, the author even suggests running
(instead of the much simpler `ls junk*`. Do I need to point that out?).I also know some people are going to retort that wasting a process or a pipe is not that bad. This discussion has been had over and over again, probably ever since Unix was invented.
In any case, the poor example chosen, as well as the flaky commands used, seriously hinder the credibility of the article.
[1]: http://mywiki.wooledge.org/ParsingLs
edit: I acknowledge the points in your link. Advocates for the unix "bunch of small tools" philosophy tend to overlook its weak spots.
Also ls is never aliased to "ls --color=always" unless you've done that more likely is its aliased to "ls --color=auto" or similar which will work normally.
My ls (which I haven't much reconfigured) behaves very differently depending on whether it's outputting to a shell or a pipe or redirect. Not just colours, but columning and filetype indicators as well.
But there's quite a lot of nice non-standard options in the GNU userspace tools, and it would be a shame not to use them when available.
Actually it does not run command once for each line of input. This confused me for some time until I read the man page:
"The command line for command is built up until it reaches a system-defined limit (unless the -n and -L options are used). The specified command will be invoked as many times as necessary to use up the list of input items. In general, there will be many fewer invocations of command than there were items in the input. This will normally have significant performance benefits. Some commands can usefully be executed in parallel too; see the -P option"
Think you're looking for "-n 1" if you want that behavior..
[0]: http://www.gnu.org/software/parallel/
My favorite usage of xargs if for parallel operations: cat hostlist | xargs -P 0 -n 1 -I host rsh host gzip -9 log/*.log
https://github.com/andreafrancia/trash-cli/#readme
As for the points about a bad example, I am fully aware that it is extremely contrived. The point was to not confuse anyone with commands they don't already know. I'm wary of posting examples that necessitate people learning 4-5 new commands when their focus is learning just one.