[Solved] “bash: Argument list too long” Error

The “Argument list too long” is an error in Bash shell. Users often encounter the error while executing commands with too many arguments. This error particularly occurs while attempting to delete, move, or copy too many files at once. The aim of this article is to explain the reason that triggers the “Argument list too long” error. Moreover, the article will discuss the available solutions to overcome the error.

What is “Argument list too long” Error in Bash?

The “Argument list too long“ error in Bash occurs when attempting to execute a command that involves passing a large number of arguments. This error usually arises when the total length of arguments passed to a command exceeds a certain limit. This limit is determined by the shell’s buffer size. If the number of arguments passed to a command is greater than the argument buffer space then Bash can’t handle the long list of arguments and raises the “Argument list too long” error.

Scenarios That Trigger Bash “Argument list too long” Error

Various scenarios can trigger the “Argument list too long” error in Bash. However, the root cause remains the same for all cases. Even users can end up with this error while using popular commands such as rm, cp, mv etc.

To simulate the error and show you the reason for the error, a folder temp is created using the following command:

mkdir temp

All the demonstrations of this article are shown in the temp folder. Now, change the current directory to the temp folder using the cd command:

cd temp

At this point, a huge number of files let’s say- 240k are created with the following Bash script:

#!/bin/bash

for i in {1..240000}; do
 touch file"$i".txt;
done

Check how many files are there in the temp directory with the following command:

ls -lrt | wc -l

This command lists the contents of the current directory in a long format, sorted by modification time with the newest files last, and then counts the number of lines in that output. This effectively gives you the total number of files (including sub-directories) in the current directory.

Now, deleting all the files of this directory using the rm command raises the “Argument list too long” error. For instance:

rm *.txt

This happens because the shell expands * to match all files (“file1.txt”, “file2.txt” … “file240000.txt”), and the resulting command becomes:

rm file1.txt file2.txt file3.txt file4.txt ... file240000.txt

In this case, the argument list becomes equivalent to the number of files in the temp directory, which is 240,000 files. This results in a long list of arguments and crosses the limit of arguments.

To get the limit of arguments that the current system allows, utilize the getconf command:

getconf ARG_MAX

It doesn’t give the number of arguments directly but provides the maximum length of the arguments that can be passed to a command in bytes. The output indicates the maximum size of all arguments combined for the current system is 2097152 bytes.

3 Ways to fix “Argument list too long” Error in Bash

There are multiple ways to overcome the “Argument list too long” error in Bash. You can use the xargs command and for loop to make the argument list shorter for each execution. Moreover, manually splitting the long list of arguments also fixes the problem. The following discussion explains some of the solutions in detail.

1. Using the “xargs” Command

To fix “Argument list too long” error in Bash, use the xargs command. For instance, if you want to delete a long list of files at a time, simply the rm command can’t handle it. However, the xargs command can execute commands such as the rm command on chunks of files, avoiding the “Argument list too long” error. Look at the example of deleting 240000 files:

find . -iname "file*" | xargs rm

Here, the find command is used to match 240000 files from “file1.txt” to “file240000.txt”. After that, these files are piped to xargs to execute the rm command on each file. After executing the command, the temp folder becomes empty and the ls command can’t find anything to list.

Learn that, xargs --show-limits displays the system limits for the number of arguments and the maximum size of the command line that xargs can handle. This information can be helpful for understanding the limitations when using xargs to process commands with a large number of arguments. For example:

xargs –show-limits

The image shows lots of information. Most importantly it displays that the ‘upper limit on argument length’ is 2091737 bytes for this system. It means Bash can allocate 2091737 bytes at max for arguments of a command while using xargs.

2. With a For Loop

For loop is another way to fix the “Argument list too long” error in Bash. Think about the files of temp directory. Instead of deleting all the files at once, a for loop can iterate over the files and delete them one by one as follows:

#!/bin/bash

for f in file*; do
 rm "$f"
done
echo "All the files are deleted."

After executing the script, the ls command can’t find any file that matches the pattern “file*”.  Though this approach of handling a long list of arguments is slower, it effectively fixes the issue.

3. By Manual Splitting

A brute-force approach to avoid the “Argument list too long” error in Bash involves manually splitting the arguments into smaller batches. For instance, “file1.txt” to “file240000.txt” can be divided into four groups and necessary commands can be executed on the smaller bunches.

Check below how manual splitting avoids “Argument list too long” error while deleting 240k files from the temp folder:

rm file{1..60000}.txt
rm file{60001..120000}.txt
rm file{120001..180000}.txt
rm file{180001..240000}.txt

Though a single rm command fails to delete 240k files at a time, dividing the files into four groups and executing rm on those removes all the files from the directory.

Remove All the Content of a Directory

In case deleting all the files of a directory causes the “Argument list too long” error, an alternative solution is removing the entire directory instead of deleting the files. For example, delete the already created temp folder and recreate it again with the following commands:

cd .. && rm -rf /home/laku/temp
cd home/laku && mkdir temp && home/laku/temp

Both commands use && to execute two commands at once. The first command changes the current directory to the parent directory of temp and then removes the directory. Then the second command creates the temp folder again and changes the current directory to temp. This way all the files of the temp folder have been deleted and a new temp folder has been created.

Conclusion

To sum up, the “Argument list too long” error is a common issue in Bash while working with a large number of arguments passed to a command. Its solutions are pretty straightforward once you understand the reason behind it. You can handle the error effectively using commands such as “xargs” or employing loops to process the arguments one by one. I believe that from now on, you can easily choose the proper solution to handle the error when you encounter it.

People Also Ask

How to simulate the “Argument list too long” error in Bash?

To simulate the “Argument list too long” error in Bash, you can use brace expansion to create a large number of files with sequential names. For instance, try to execute the following command to create 240k files:

touch file{1..240000}.txt

This will raise “Argument list too long” error. If it doesn’t, gradually increase the range until the error occurs.

How do you resolve the “Argument list too long” error in Bash?

To resolve the “Argument list too long” error in Bash, you have a few options:

1. xargs command: Instead of passing all arguments directly to a command, you can use the xargs command to pass them in manageable chunks.
2. loop: Use Bash for loop to iterate over each argument and perform the operation on each file individually.
3. Manual splitting: Manually split the argument list into multiple groups and perform the required operation on each group.

How do I avoid “Argument list too long” errors when compressing thousands of files?

You can avoid “Argument list too long” errors when compressing thousands of files by using the find command. Here’s an example:

find . -name "*.xml" -print | tar -czvf xml.tgz -T -

This command will find all files with a .xml extension in the current directory and its subdirectories, safely pass their names to tar, and then compress them into xml.tgz.

How do I delete too many files in Bash?

To delete too many files in Bash, you can use the find command with exec. Here’s a general example:

find /path/to/files -type f -name "pattern" -exec rm {} +

Replace /path/to/files with the directory containing the files you want to delete and “pattern” with the filename pattern or expression you want to match. This command will find all files matching the pattern and delete them efficiently.

What is the maximum number of arguments in a Bash script?

There’s no single, definitive answer to the maximum number of arguments in a Bash script. Bash has a limitation on the overall length of arguments of a command. You can use the getconf ARG_MAX to find the maximum size of arguments in bytes.

What is the exact limit of arguments to avoid “Argument list too long” error?

The exact limit of arguments to avoid the “Argument list too long” error can vary depending on the system. However, a common limit is typically around 128 KB to 2 MB for total argument length. This limit includes not only the length of each individual argument but also the overhead of the command itself and any environment variables. It’s important to note that this limit is not a fixed number of arguments but rather a limit on the total length of all arguments combined. Therefore, the actual number of arguments that will trigger the error can vary based on the length of each argument.

Related Articles

• How to Check Number of Arguments in Bash? [3 Methods]
• How to Use First Argument in Bash Script [5 Cases]
• How to Get Argument in Bash? [4 Methods]
• How to Pass Arguments to Bash Script? [5 Methods]
• How to Parse Optional Arguments in Bash Script [2 Ways]
• How to Pass Array as an Argument in Bash Function? [3 Ways]
• How to Set Default Argument in Bash [2 Methods]
• How to Use OPTARG in Bash [3 Practical Examples]
• How to Use “getopts” in Bash [Complete Guide]

---

<< Go Back to Argument in Bash Script | Bash Functions | Bash Scripting Tutorial