How to Run Bash Commands in Parallel [4 Ways With Examples]

Running Bash commands in parallel is a technique or method of executing multiple commands simultaneously in a Bash shell environment. This method helps in reducing overall execution time. In this article, I will discuss different ways to run multiple Bash commands in parallel. I will also show some practical examples with Bash scripts of parallel operations. So let’s start!

4 Ways to Run Bash Commands in Parallel

Read out the following described ways, through which you can run multiple Bash commands of a Bash script in parallel while executing the script.

1. Ampersand “&” Sign

As a simple approach, use the inherent Bash ampersand (&) operator to run commands in parallel. It allows the execution of a command asynchronously, enabling the shell to proceed to the next command without waiting for the current one to finish.

Basic Syntax:

command1&
command2&

2. The “wait” Command With the Ampersand “&” Sign

The wait command waits for all child processes to exit. So using the wait command with the ‘&’ operator we can run batches of operations. However, the one drawback of using the wait command is that you can’t get a new process to start as soon as a running process ends.

Basic Syntax:

command1&
command2&
wait

3. The “xargs” Command

The xargs command can be used to execute commands in parallel by combining it with the -p option. This option specifies the maximum number of parallel processes to run.

4. GNU Parallel

GNU Parallel is a powerful tool that allows you to run commands in parallel, providing flexibility and control over the execution process. You can install GNU Parallel on your system directly from the command line. By default, the parallel is not included & needs to be installed.

Basic Syntax:

parallel ::: prog1 prog2

Here, ::: is a special syntax that tells the command to iterate over a list of arguments.

3 Practical Examples of Running Bash Commands in Parallel

In the following article, I will explain three practical examples of running Bash commands in parallel, based on the above-mentioned ways.

Example 1: Using “&” Sign With “wait” Command

In the first example, I will run a Bash script with multiple commands. Inside the script, I will use the “&” sign along with the wait command to perform parallel operations among those written Bash script commands.

To run commands in parallel using the ampersand & with the wait command, see the following Bash script:

#! /bin/bash

#Command 1 running in the background
uname&
#Command 2 running in the background
date&
#Command 3 running in the background
ls -l&

#wait for all background processes to complete
wait

echo "All commands completed"

EXPLANATION
In  the first script line (#! /bin/bash), #! is called shebang or hashbang. It indicates the interpreter to be used for executing the script, in this case, it’s Bash. Next, 3 commands are in 3 lines along with the ampersand sign to be executed in parallel.

Afterward, the wait command waits for all the background processes to complete. When the processes are complete the echo command prints the completion message.

Run bash commands in parallel with '&' operatorAfter executing, you can see the script has executed all the commands parallelly & displaying the outputs one by one.

Example 2: Run Commands in Parallel Using GNU Parallel Command Tool

Here, I will use the powerful GNU command tool parallel to run several Bash commands within a script in parallel. Now, this command tool is not installed by default. However, you can install it on your Ubuntu by running the following command:

sudo apt install parallel

To run commands in parallel using the GNU command parallel, check out the following Bash script:

#! /bin/bash

commands=(
      "date"
       "uname"
       "uptime"
)
parallel --jobs 3 ::: "${commands[@]}"

echo "All commands completed"

EXPLANATION
The script starts with an array called commands that holds the commands to be executed in parallel.

After that, the parallel command with the option --jobs 3 specifies that a maximum of 3 jobs should be executed concurrently. The ::: syntax followed by "${commands[@]}" passes the elements of the ‘commands’ array as arguments to ‘parallel’.

Run bash commands in parallel with GNU parallel toolThe script prints all three commands’ output upon execution along with the All commands completed message after all the operations are done.

Example 3: Run Commands in Parallel Using the “xargs” Command

In this example, I will use the xargs command in a Bash script to run some commands in parallel. To run commands in parallel using the xargs command, please go through the below script:

#! /bin/bash

commands=(
    "ls -l"
    "date"
    "whoami"
)

printf "%s\n"  "${commands[@]}" | xargs -I {} -P 3 sh -c "{}"
echo "All commands completed"

EXPLANATION
The three commands ls -l, ncal, and whoami in the script list files & directories, prints date, & display username respectively. Now when you run the script these commands will be executed in parallel using the xargs command with the -P option where 3 specifies that a maximum of 3 parallel processes should be running at any given time. The -I {} option replaces occurrences of {} in the command with the input.

The printf "%s\n"  "${commands [@]}" command prints each command output on a separate line. The output is then piped | to xargs, which reads the input lines & executes the command sh -c "{}" in parallel. And, this command instructs the shell interpreter to execute the command represented by {} will be replaced by each command from the array as xargs processes them.

Run bash commands in parallel with the xargs command The script prints all three commands’ output upon execution along with the All commands completed message after all the operations are done.

Importance of Running Bash Commands in Parallel

Check out the significant outcomes you can get while running several Bash commands in parallel:

  • By running commands in parallel, you can take the benefits of multicore processes and execute tasks concurrently. And that reduces overall execution time.
  • When writing scripts or automation takes place, running commands in parallel within Bash scripts can increase efficiency ensuring faster execution.
  • You can process a large number of files and even a complex command that takes a long time to execute.
  • Improve the performance of a Bash script that is running on a single CPU.

Conclusion

In conclusion, learning to run several Bash commands in parallel can enhance your Bash scripting experience with speed, performance, & efficiency. Using some simple commands you can make it possible. I hope this article helps you learn the entire process.

People Also Ask

Does bash run in parallel?

Bash is primarily designed for the sequential execution of commands. That means, out of multiple commands, the first one will be executed first & Bash will wait for it to end before the next command. However, using certain operators & commands you can commands in parallel within a Bash script.

How do I run two shell commands in parallel?

To run two shell commands in parallel, you can use the and & operator to start a command in the background. This allows both commands to run simultaneously. For example, use command syntax, command1 & command2 &.

How to do parallel execution in bash script?

To achieve parallel execution of multiple commands within a Bash script, you can use the ‘&’ operator to start each command in the background and then use the ‘wait’ command to wait for all background processes to finish.

How do I run a bash command on multiple lines?

In Bash, you can run a command on multiple lines by using a backlash \ to indicate line continuation. Moreover, this allows you to break a long command into multiple lines for better readability.


Related Articles


<< Go Back to Executing Bash Script | Bash Scripting Basics | Bash Scripting Tutorial

4.7/5 - (15 votes)
LINUX
FUNDAMENTALS
A Complete Guide for Beginners Enroll Course Now
Monira Akter Munny

Hello!! This is Monira Akter Munny. I'm a Linux content developer executive here, at SOFTEKO company. I have completed my B.Sc. in Engineering from Rajshahi University of Engineering & Technology in the Electrical & Electronics department. I'm more of an online gaming person who also loves to read blogs & write. As an open-minded person ready to learn & adapt to new territory, I'm always excited to explore the Linux world & share it with you! Read Full Bio

Leave a Comment