How to Use the xargs Command on Linux

Here we can see, “How to Use the xargs Command on Linux”

Need to chain together a few Linux tasks, but one of them won’t take piped input? xargs can take the output of one command and give it as arguments to another programme.

There are three data streams connected with all of the basic Linux utilities. The standard input stream (stdin), the standard output stream (stdout), and the standard error stream (stderr) are the three streams (stderr).

Text is used in these streams. For example, we use text to send input (stdin) to the command and write the answer (stdout) to the terminal window. Error messages are also shown as text in the terminal window (stderr).

The ability to pipe the stdout output of one command into the stdin input of another programme is one of the best features of Linux and Unix-like operating systems. The first command is unconcerned that its output does not go to a terminal window, while the second command is unconcerned that its input does not come from a keyboard.

Even though all Linux commands have three standard streams, not all accept stdout from another command as input to their stdin. That implies you won’t be able to stream data to them.

xargs is a command that allows you to create execution pipelines from ordinary data streams. For example, we can make programmes like echo, rm, and mkdir take standard input as arguments by using xargs.

The xargs Command

Xargs support piped input. It can also read information from a file. Xargs use that input as arguments for the commands we’ve given it. If we don’t instruct xargs to use a specific command, it will use echo by default.

We can use this to show how, even with multi-line input, xargs always produces a single line of output.

We receive a single column of filenames if we use the -1 (list one file per line) option with ls.

ls -1 ./*.sh

This command displays a list of all shell script files in the current directory.

As expected, we only got one column. So what exactly do we get if we run it through xargs?

ls -1 ./*.sh | xargs

The output is written as a continuous stream of text to the terminal window.

This is how xargs can feed parameters into other commands.

Using xargs With wc

We can easily have wc count the words, characters, and lines in multiple files by using xargs.

ls *.page | xargs wc

What happens is this:

• ls outputs a list of *.page files, which xargs parses.
• Filenames are passed to wc by xargs.
• The filenames are treated as if they were command-line arguments by wc.

Each file’s statistics are displayed alongside an overall total.

Using xargs With Confirmation

We can use the -p (interactive) option to have xargs ask for confirmation before proceeding.

Touch will create files for us if we pass a string of filenames to it via xargs.

echo 'one two three' | xargs -p touch

The command that will be executed is displayed, and xargs waits for us to type “y” or “Y” or “n” or “N”, and press Enter to respond.

• It is treated as “n” if you press Enter. So, for example, if you type “y” or “Y,” the command will be executed.
• We typed “y” and then “Enter.” To verify that the files have been created, we can use ls.

ls one two three

Using xargs With Multiple Commands

Using the -I (initial arguments) option, we can run multiple commands with xargs.

This option defines a “replace-string”. The values supplied to xargs are inserted wherever the token for the replace-string appears in the command line.

Let’s look at the subdirectories in the current directory with the tree command. Tree ignores files and only reports on directories when the -d (directory) option is used.

tree -d

The “images” subdirectory is the only one.

We have the names of some directories that we want to create in a file called “directories.txt.” Then, we can use a cat to examine its contents.

cat directories.txt

We’re going to use this as the xargs input data. So this is the command we’ll use:

cat directories.txt | xargs -I % sh -c 'echo %; mkdir %'

This is how it breaks down:

• Cat directories.txt |: This populates xargs with the contents of the directories.txt file (all the new directory names).
• xargs -I %: This defines a “replace-string” with the token “%”.
• This command creates a new subshell. The -c (command) option instructs the shell to read commands from the terminal.
• ‘echo %; mkdir %’: each of the “%” the directory names passed by xargs will replace each “percent” token. The directory name will be printed with the echo command, and the directory will be created with the mkdir command.

One by one, the directories are listed.

We can use the tree to verify that the directories have been created once more.

tree -d

Copying Files To Multiple Locations

With xargs, we can copy files to multiple locations with just one command.

The names of two directories will be piped into xargs as input parameters. We’re going to tell xargs only to pass one of these parameters to the command it’s working with at a time.

The command is cp in this case. As a result, cp is called twice, with one of the two directories as a command-line parameter each time. The -n (max number) xargs parameter is what allows this to happen. We’re going to make this one of them.

We’re also running cp with the -v (verbose) option to see what’s going on.

echo ~/Backups/ ~/Documents/page-files/ | xargs -n 1 cp -v ./*.page

One directory at a time, the files are copied to the two directories. cp logs each file copy operation so we can see what’s going on.

Deleting Files in Nested Directories

xargs will not interpret filenames that contain spaces or unusual characters, such as newline characters. The -0 (null terminator) option can be used to solve this problem. This instructs xargs to use the null character as the filename’s final delimiter.

In this example, we’ll use the find command. For dealing with whitespace and strange characters in filenames, find has its option. It’s the option -print0 (full name, null character).

find . -name "*.png" -type f -print0 | xargs -0 rm -v -rf "{}"

This breaks down like this:

• find .-name “*.png”: find will look for objects with names that match “*.png” that are files in the current directory “.” (type -f).
• -print0: names are terminated with a null character, and spaces and unusual characters are accommodated.
• xargs -0: xargs will also treat filenames as null-terminated, so spaces and strange characters will not cause issues.
• rm -v -rf “{}”: rm will be verbose and report on what is going on (-v). It will be recursive (-r), searching nested subdirectories and removing files without prompting (-f). Each filename takes the place of the “.”

The files that match the search pattern are deleted from all subdirectories.

Removing Nested Directories

Let’s pretend we want to get rid of a bunch of nested subdirectories. We’ll be able to see them because of the tree.

tree -d

find . -name "level_one" -type d printo | xargs -o rm -v -rf "{}"

This command will use find to search the current directory recursively. The directory “level one” is the search target. xargs is used to pass directory names to rm.

The only difference between this and the previous command is that the search term is the name of the topmost directory, and -type d instructs find to look for directories rather than files.

Each directory’s name is printed as it is removed. We can double-check with the tree:

tree -d

The nested subdirectories are all removed.

Deleting All Files, Except for One File Type

find, xargs, and rm can delete all files except one type that we want to keep. We provide the name of the file type we want to keep, not the name of the ones we want to delete, which is a little counterintuitive.

Find is told to return the names of files that don’t match the search pattern with the -not option. With xargs, we’re using the -I (initial arguments) option once more. The replace-string token we’re defining this time is”  “. This will behave identically to the replace-string token we created earlier, which happened to be a “percent “.

find . -type f -not - name "*.sh" -print0 | xargs -0 -I {} rm -v {}

We can use ls to check. Only the files that matched the “*.sh” search pattern remain in the directory.

ls -l

Creating an Archive File With Xargs

To create an archive file, we can search for files and then pass them through xargs to tar.

We’ll look through the current directory. We’ll be looking for “.page” files because the search pattern is “*.page.”

find ./ - name "*.page" -type f -print0 | xargs -0 -tar -cvzf page_files.tar.gz

As the archive file is created, the files are listed as expected.

The Data Mediator

When you’re stacking things together, you may require some scaffolding. xargs is a command that bridges the gap between commands that can output data and commands that aren’t designed to take it in.

Conclusion

I hope you found this information helpful. Please fill out the form below if you have any questions or comments.

User Questions:

1. What does xargs command do?

On Unix and most Unix-like operating systems, xargs (short for “eXtended ARGuments”) is a command that builds and executes commands from standard input. It takes standard input and converts it into command arguments.

2. Does xargs run in parallel?

xargs will run the first two commands in parallel, and when one of them finishes, it will start the next one until the job is completed. The same concept can be applied to as many processors as you have at your disposal. It also applies to resources other than processors.

3. What is bash xargs?

In a UNIX shell, the xargs command converts input from standard input into command arguments. In other words, xargs allow one command’s output to be used as the input of another. For example, that will run the command2 command, which will take the output of command1 as an argument (s).

4. Convert parallel to xargs command

Convert parallel to xargs command from linuxquestions

5. Using the Shell: xargs

Using the Shell: xargs from commandline