#10: Using xargs
One very helpful tool that exists on many Unix-like systems is xargs. xargs is built to receive standard input (STDIN) and pass it as command-line parameters to a specified command. Because most UNIX systems have a built-in ARG_MAX constant, which limits the length of command-line parameters, xargs splits the input into as many parts as needed and passes them separately to the given command. That, by the way, is the main purpose of xargs but since Linux has removed ARG_MAX in 2.6.23, it's no longer needed for that on this platform.
But xargs can do much more for you than just avoiding the ARG_MAX limit. For instance, you can format command-line arguments. Unlike the normal backtick operator, xargs is much more flexible in regard to the format of STDIN and what the command, which the arguments are passed to, needs. For example, when you list the contents of a directory and the file names contain spaces, the backtick operator would fail.
rm `find . -type f`If one of the file names contains whitespace, you may run into big trouble. However, xargs can manage this quite well. find and xargs are actually a great team. To work around the problem above, we just make find using a different delimiter for separating its output and xargs dividing the arguments by even that delimiter character. The best character to accomplish this is the NULL byte \0. The following code snippet passes a NULL byte separated file list via pipe to xargs, which runs rm for all these files:
find . -type f -print0 | xargs -0 rmThe parameters -print0 and -0, respectively, make both commands work with NULL bytes so you don't have to worry about spaces anymore.
But xargs can do even more. You can also specify any other delimiter with -d. One other great trick I want to show you is the possibility of executing commands interactively. For this rm is a bad example because it already has an interactive mode (parameter -i) so let's use touch. Let's assume that you want to touch some TXT files in your current working directory but not all of them. To get this done we instruct xargs to limit the maximum number of arguments per command execution to 1, which will then run the specified command for each single argument.
find . -maxdepth 1 -type f -name "*.txt" -print0 | xargs -0 -p -n 1 touchThis lists all *.txt files in the current directory and passes them to xargs (separated by NULL bytes), which asks (-p) for each single argument (-n 1 limits the number of per-command arguments to one), whether you would like to touch this file or not.
These are a few popular use cases for xargs. If some others cross your mind, feel free to share them with other readers in the comments section of this article. I always appreciate feedback.
Read more about xargs:
Trackbacks
Comments
find . -type f -print0 | xargs -0 rm
-> or do the same with -exec (no worries about spaces too): find . -type f -exec rm {} \;
Never heared from ARG_MAX, but you are just right, xarg rocks :)
Thanks for your addition. The exec parameter of find is indeed interesting, but personally I would prefer xargs over exec. xargs is more efficient because it runs the command as few times as possible, whereas find would run the command for each file name. See "here":http://danielmiessler.com/blog/linux-xargs-vs-exec for more information about this.
However, modern versions of find implement this:
pre. find . -type f -exec rm ‘{}’ +
which does the same as xargs (note the + instead of the \;), but since not all systems might have a modern version of find, xargs is the more portable variant.
find -print0 isn’t portable either, it’s a GNU thing… Which is pretty frustrating IMO, since a straight up "find | xargs" chokes on whitespace…
I suppose "find | xargs —delimiter ‘\n’" also works (as long as there’s not a newline in a filename… what kind of sick bastard would do that anyway?) - don’t know offhand if "—delimiter" is also GNU specific or more cross-implementation reliable…
Personally I never use "find —exec" because I always found the syntax too convoluted for anything non-trivial…
With portable I meant the compatibility between different GNU compatible systems.
I don’t know how much this is GNU specific, but instead of -print0 you could also use -printf "%p\0".
RT @reflinux: #Advent series "24 Short #Linux #Hints", day 10: Using #xargs http://bit.ly/fiUaPU