News for hackers. Stuff that matters.
Re:Bash process substitution by skywhopper on Sunday February 08, 2015 @07:24PM (#9019175)
I use this a lot in a couple of different contexts.

First, it's a convenient way to turn around pipelines inside scripts in situations where you need to feed a command output or pipeline into a read loop but the loop needs to be part of the original script process so it can manipulate variables, etc. Meaningless example of optionally incrementing a counter in a loop based on the output of a pipeline of commands:

    while read LINE; do
        if grep -q xyz <<< "$LINE"; then
            COUNTER=$(( $COUNTER + 1 ))
        fi
    done < <(awk '{print $4,$7}' infile.txt |cut -d, -f3)
If you did it in the "awk ...|cut ...| while read LINE..." order, COUNTER would be getting incremented in a subshell, and the value wouldn't stick around after the loop.

The other most common way I use it is to diff a couple of sets of pipelines or even just files that need sorting before comparison:

    diff <(sort -u file1.txt) <(sort -u file2.txt)
Saves a ton of steps. As pointed out by jkbyc, the >() construction for output process substitution becomes really powerful when combined with tee.
Re:Bash process substitution by skywhopper on Monday February 09, 2015 @07:29AM (#9020811)
Another example of using diff on two process substitutions I did this morning. I've been quarantining some files from various paths that all have unique basenames into a single directory. I have a CSV file listing the original path in the first field followed by some other metadata. I wanted to compare the files in the CSV file ("quarantined-files.csv") to the files I had moved into the "quarantine" subdirectory. I came up with this:

    diff <(cut -d, -f1 quarantined-files.csv |xargs -L1 basename |sort) <(ls ./quarantine/ |sort)
(Yes, the listing from "ls" comes out sorted, but it's always possible it'll sort certain characters a bit differently than "sort" does depending on locale settings, aliases, etc, etc. Piping both outputs through "sort" should ensure the same sorting collation in all cases.)

It took me a while to really grok this sort of thing, but once it clicked, it's often the first thing I think of. Basically any time you might be tempted to dump something to a temporary file, you should ask yourself "can I use process substitution?"