#17: Process substitution
You probably know command substitution. Command substitution executes a command and uses its output as a variable on the shell. The operator for this is either the backtick operator
`command`or the (newer and recommended) parenthesis operator with a dollar sign:
$(command)Command substitution is very common and I use it often for things like rebuilding my input drivers after xorg updates:
emerge -av1 `eix -I --only-names x11-drivers/`or alternatively
emerge -av1 $(eix -I --only-names x11-drivers/)There are hundreds of thousands of different use cases for this. But did you also know that there is a counterpart of command substitution? This counterpart is called process substitution. Unlike command substitution this does not use the output of a command as a variable, but writes it to a FIFO in /etc/fd/ or receives its STDIN from there (I wrote about FIFOs earlier this month). Instead of using the output directly, only the filename of this FIFO/named pipe is used. The operators for process substitution are
<(command)and
>(command)The first one writes its STDOUT to the FIFO and can be used as input file for another program. The second one receives its STDIN from the FIFO. Thus, when you write to the file, the command in parentheses receives the written data as STDIN. Note that there is no space after < and >!
You can see how process substitution works by running
echo <(true)
/dev/fd/63The STDOUT of <(true) is written to the FIFO /etc/fd/63 and then the command itself has been substituted with the filename. The same happens to the second operator:
echo >(true)
/dev/fd/63But how do we use this? Process substitution is very helpful if you have a command which only accepts files and does not work with STDIN/STDOUT. For example, if you want to compare the output of two commands with diff, you could write their output into files and then compare them with diff, but there is a much easier way to do this:
diff <(echo 'foo') <(echo 'bar')The output is
1c1
< foo
---
> barSimple but handy. You could also use this with grep. Instead of piping STDOUT you could also make grep reading from a FIFO:
grep foobar <(ps auxw)Since grep can operate on STDIN as well as on files this leads to the same result as
ps auxw | grep foobarOf course the piping variant is much simpler, but this is just an example to demonstrate how process substitution works.
As described above, the second process substitution operator does not provide its STDOUT via FIFO but receives its STDIN from it. For instance, a complex and preposterous way of printing “foobar” to the screen would be:
echo 'foobar' > >(cat)This redirects the output of the echo command to a file, which is our FIFO. Then the command in parentheses receives the written data as STDIN. A more meaningful example would be to pipe STDOUT of a command to several programs at once. For instance:
echo 'foobar' | tee >(command1) >(command2) | command3In this example tee would write to the FIFOs for command1 and command2, which get the data as STDIN. Then all the STDOUT of tee, including STDOUT of command1 and command2, is passed to command3 since it's piped to tee and all its arguments. If that's not what you want you have to use process substitution for command3 as well.
Another example could be to compress files generated by a program, but this does not work in all cases. For instance, I had some trouble with mencoder, perhaps because it checks the size of the output file. If a program verifies what it has written or does some calculation based on the output file, writing to FIFOs might not work.
There are not so many cases in which the >(command) notation is expedient, but <(command) is very handy in several cases. But keep in mind that process substitution is very Bash specific. Only a few other shells have implemented it as well (such as ZSH).
Read more about process substitution:
RT @reflinux: #Advent series "24 Short #Linux #Hints", day 17: #Process substitution http://bit.ly/eC8dzE