Running Bash commands in parallel is a technique or method of executing multiple commands simultaneously in a Bash shell environment. This method helps in reducing overall execution time. In this article, I will discuss different ways to run multiple Bash commands in parallel. I will also show some practical examples with Bash scripts of parallel operations. So let’s start!
4 Ways to Run Bash Commands in Parallel
Read out the following described ways, through which you can run multiple Bash commands of a Bash script in parallel while executing the script.
1. Ampersand “&” Sign
As a simple approach, use the inherent Bash ampersand (&) operator to run commands in parallel. It allows the execution of a command asynchronously, enabling the shell to proceed to the next command without waiting for the current one to finish.
Basic Syntax:
command1&
command2&2. The “wait” Command With the Ampersand “&” Sign
The wait command waits for all child processes to exit. So using the wait command with the ‘&’ operator we can run batches of operations. However, the one drawback of using the wait command is that you can’t get a new process to start as soon as a running process ends.
Basic Syntax:
command1&
command2&
wait3. The “xargs” Command
The xargs command can be used to execute commands in parallel by combining it with the -p option. This option specifies the maximum number of parallel processes to run.
4. GNU Parallel
GNU Parallel is a powerful tool that allows you to run commands in parallel, providing flexibility and control over the execution process. You can install GNU Parallel on your system directly from the command line. By default, the parallel is not included & needs to be installed.
Basic Syntax:
parallel ::: prog1 prog2Here, ::: is a special syntax that tells the command to iterate over a list of arguments.
3 Practical Examples of Running Bash Commands in Parallel
In the following article, I will explain three practical examples of running Bash commands in parallel, based on the above-mentioned ways.
Example 1: Using “&” Sign With “wait” Command
In the first example, I will run a Bash script with multiple commands. Inside the script, I will use the “&” sign along with the wait command to perform parallel operations among those written Bash script commands.
To run commands in parallel using the ampersand & with the wait command, see the following Bash script:
#! /bin/bash
#Command 1 running in the background
uname&
#Command 2 running in the background
date&
#Command 3 running in the background
ls -l&
#wait for all background processes to complete
wait
echo "All commands completed"#! is called shebang or hashbang. It indicates the interpreter to be used for executing the script, in this case, it’s Bash. Next, 3 commands are in 3 lines along with the ampersand sign to be executed in parallel.
Afterward, the wait command waits for all the background processes to complete. When the processes are complete the echo command prints the completion message.
After executing, you can see the script has executed all the commands parallelly & displaying the outputs one by one.
Example 2: Run Commands in Parallel Using GNU Parallel Command Tool
Here, I will use the powerful GNU command tool parallel to run several Bash commands within a script in parallel. Now, this command tool is not installed by default. However, you can install it on your Ubuntu by running the following command:
sudo apt install parallelTo run commands in parallel using the GNU command parallel, check out the following Bash script:
#! /bin/bash
commands=(
"date"
"uname"
"uptime"
)
parallel --jobs 3 ::: "${commands[@]}"
echo "All commands completed"commands that holds the commands to be executed in parallel.
After that, the parallel command with the option --jobs 3 specifies that a maximum of 3 jobs should be executed concurrently. The ::: syntax followed by "${commands[@]}" passes the elements of the ‘commands’ array as arguments to ‘parallel’.
The script prints all three commands’ output upon execution along with the All commands completed message after all the operations are done.
Example 3: Run Commands in Parallel Using the “xargs” Command
In this example, I will use the xargs command in a Bash script to run some commands in parallel. To run commands in parallel using the xargs command, please go through the below script:
#! /bin/bash
commands=(
"ls -l"
"date"
"whoami"
)
printf "%s\n" "${commands[@]}" | xargs -I {} -P 3 sh -c "{}"
echo "All commands completed"-P option where 3 specifies that a maximum of 3 parallel processes should be running at any given time. The -I {} option replaces occurrences of {} in the command with the input.
The printf "%s\n" "${commands [@]}" command prints each command output on a separate line. The output is then piped | to xargs, which reads the input lines & executes the command sh -c "{}" in parallel. And, this command instructs the shell interpreter to execute the command represented by {} will be replaced by each command from the array as xargs processes them.
The script prints all three commands’ output upon execution along with the All commands completed message after all the operations are done.
Importance of Running Bash Commands in Parallel
Check out the significant outcomes you can get while running several Bash commands in parallel:
- By running commands in parallel, you can take the benefits of multicore processes and execute tasks concurrently. And that reduces overall execution time.
- When writing scripts or automation takes place, running commands in parallel within Bash scripts can increase efficiency ensuring faster execution.
- You can process a large number of files and even a complex command that takes a long time to execute.
- Improve the performance of a Bash script that is running on a single CPU.
Conclusion
In conclusion, learning to run several Bash commands in parallel can enhance your Bash scripting experience with speed, performance, & efficiency. Using some simple commands you can make it possible. I hope this article helps you learn the entire process.
People Also Ask
Does bash run in parallel?
Bash is primarily designed for the sequential execution of commands. That means, out of multiple commands, the first one will be executed first & Bash will wait for it to end before the next command. However, using certain operators & commands you can commands in parallel within a Bash script.
How do I run two shell commands in parallel?
To run two shell commands in parallel, you can use the and & operator to start a command in the background. This allows both commands to run simultaneously. For example, use command syntax, command1 & command2 &.
How to do parallel execution in bash script?
To achieve parallel execution of multiple commands within a Bash script, you can use the ‘&’ operator to start each command in the background and then use the ‘wait’ command to wait for all background processes to finish.
How do I run a bash command on multiple lines?
In Bash, you can run a command on multiple lines by using a backlash \ to indicate line continuation. Moreover, this allows you to break a long command into multiple lines for better readability.
Related Articles
- How to make a File Executable in Bash [2 Methods]
- A Complete Overview of Bash Dot Command [With 2 Examples]
- What are the Usages of Bash Source [3 Practical Examples]
- How to Run a Bash Script [2 Methods with Cases]
<< Go Back to Executing Bash Script | Bash Scripting Basics | Bash Scripting Tutorial
FUNDAMENTALS A Complete Guide for Beginners
