Re:Bash process substitution
by skywhopper on Monday February 09, 2015 @07:29AM
(#9020811)
Another example of using diff on two process substitutions I did this morning. I've been quarantining some files from various paths that all have unique basenames into a single directory. I have a CSV file listing the original path in the first field followed by some other metadata. I wanted to compare the files in the CSV file ("quarantined-files.csv") to the files I had moved into the "quarantine" subdirectory. I came up with this:
diff <(cut -d, -f1 quarantined-files.csv |xargs -L1 basename |sort) <(ls ./quarantine/ |sort)
(Yes, the listing from "ls" comes out sorted, but it's always possible it'll sort certain characters a bit differently than "sort" does depending on locale settings, aliases, etc, etc. Piping both outputs through "sort" should ensure the same sorting collation in all cases.)It took me a while to really grok this sort of thing, but once it clicked, it's often the first thing I think of. Basically any time you might be tempted to dump something to a temporary file, you should ask yourself "can I use process substitution?"
First, it's a convenient way to turn around pipelines inside scripts in situations where you need to feed a command output or pipeline into a read loop but the loop needs to be part of the original script process so it can manipulate variables, etc. Meaningless example of optionally incrementing a counter in a loop based on the output of a pipeline of commands:
If you did it in the "awk ...|cut ...| while read LINE..." order, COUNTER would be getting incremented in a subshell, and the value wouldn't stick around after the loop.The other most common way I use it is to diff a couple of sets of pipelines or even just files that need sorting before comparison:
Saves a ton of steps. As pointed out by jkbyc, the >() construction for output process substitution becomes really powerful when combined with tee.