Bash Redirection and Piping

LINUX
FUNDAMENTALS
A Complete Guide for Beginners Enroll Course Now

Bash, a popular shell in Unix-like systems, provides two essential features for managing the flow of data between commands and files. These are redirection and piping. From this writing, you will learn about the basics of Bash redirection and piping, their differences & cases where you can apply them while Bash scripting.

Redirection in Bash

Redirection is a process that allows you to change the default input source or output destination of a command. With redirection, you can redirect output to a file instead of displaying it on the terminal, or instruct an application to read input from a file rather than the keyboard. You can redirect your used script commands in different ways whenever necessary. So, the following are the ways of redirection available in Linux:

A. Redirecting Input

Redirecting input is a process where the standard input (stdin) of a command is changed from the keyboard to a  file. By, using the ‘<’ symbol you can take input data from a file instead of typing it interactively. For example,

cat < file.txt

B. Redirecting Output

Here, the displaying of the standard output (stdout) of a command is changed from the terminal screen to a file or another command’s input. The ‘>’ symbol is used for this purpose. For example:

ls > output.txt

C. Redirecting Standard Error

The standard error (stderr) is a way to control where error messages, generated by commands are sent. You can redirect stderr to a file, discard it, or merge it with the standard output. You can use the redirection operator “2>” to redirect the standard error of a command to a file, “2>>” to append, or “2> /dev/null” to discard it. For example,

whoami -l 2> output.txt

D. Appending Redirected Output

Normally, when you redirect a stdout to a file with the ‘>’ redirection operator, it overwrites the file. You can also append the stdout to the existing file contents while redirecting by using the ‘>>’ redirection operators. For example,

ls >> output.txt

E. Redirecting Standard Output and Standard Error

In Linux, every process has two default output streams, stdout (represented by file descriptor 1) & standard error or stderr (represented by file descriptor 2). You can redirect both of them using the symbol ‘&> or ‘>&’ or ‘2>&1’. For example,

ls /nonexistent_directory &> log.txt

F. Appending Standard Output and Standard Error

In the same way as appending stdout, you can append both stdout & stderr at the same time to the same file too. For that, use the append symbol ‘>>’. For example,

ls >> output.txt 2>&1

Piping in Bash

Bash piping is a powerful process that allows you to connect multiple commands in Linux and other Unix-like shell environments. In a more simple context, a pipe redirects the standard output from one command to another for processing. By using the pipe character ‘|’, you can connect one command’s output to another command’s input.

Pipes are unidirectional, flowing data from left to right through the pipeline.

The Command Syntax >

Command 1 | command 2 | … | Command N

Difference Between Bash Redirection & Piping

Some users may find the operations of Bash redirection & piping quite similar. However, they have some core differences as they serve different purposes. Read out the following key points of their differences:

  • A pipe (|) directs the output of one program to another program or utility. Whereas, redirection (‘>’, ‘>>’, ‘<’, etc) controls where the output goes or comes from.
  • Pipe facilitates seamless data flow between commands in a sequence. And, redirection makes sure that output is redirected to a file (‘>’) or read from a file (‘<’).
  • Piping is unidirectional, sending data from left to right whilst redirection handles input & output separately.
  • Each serves a distinct purpose. While pipe does is collaboration, redirection emphasizes controlling inputs & outputs.

Let me simplify the two cases a bit more with the following example:

Program 1 > Program 2 vs Program 1 | Program 2

‘Program 1 > Program 2’

As soon as you execute it, your shell initiates the execution of the program labeled as ‘Program 1’. All output generated by ‘Program 1’ is stored within a file designated as ‘Program 2’ (please note, if ‘Program 2’ exists, its contents are overwritten).

‘Program 1 | Program 2’

On the other hand, if the intention is to forward the output from ‘Program 1’ to ‘Program 2’, the following approach can be used:

Program 1 > temp_file && Program 2 < temp_file

Here, first, your shell will launch ‘Program 1’, and save the output to a file name ‘temp_file’. After that, will initiate the execution of ‘Program 2’, treating the contents of ‘temp_files’ as if they were input from the keyboard.

However, this approach could be tedious. Therefore, the concept of pipes was introduced to simplify the process. So, ‘Program 1 | Program 2’ accomplishes the same objective as ‘Program 1 > temp_file && Program 2 < temp_file’.

A Sample Bash Script Example Combining Redirection and Piping

Now, let’s explore a practical example, where I will use both Bash redirection & piping processes in a single command line. To check yourself, copy the following Bash script on your nano editor & save the file with an appropriate name with the .sh extension.

Script (test.sh) >

#! /bin/bash

ls -l | grep ‘.txt’ > text_files
cat text_files

EXPLANATION
The ls command with its –long option, lists all the contents of a directory. After that, the pipe operator ‘|’ takes this output and passes it as the input for the grep command. Where, the grep command searches from those inputs for all the lines containing the specified ‘.txt’ pattern.

Afterward, the redirection operator ‘>’ redirects the standard output of the command sequence ‘ls -l | grep ‘.txt’’ to a file named ‘text_files’. If the file doesn’t exist, it will be created. If it exists, its contents will be overwritten. Finally, the cat command displays the contents of the file.

After that, use the following command to make the script executable:

chmod u+x test.sh

EXPLANATION
  • chmod: Changes the permission of files and directories.
  • u+x: Argument with chmod command to add the executable permission for the user.
  • test.sh: File which you want to make executable.

Adding executive permission

Finally, run the script by the following command:

./test.sh

Bash script example of redirection and pipingThe output of the script displays all the files with ‘.txt’ extension.

Conclusion

To sum up, Bash redirection and piping are vital Unix tools. Redirection controls I/O flow, while piping chains commands. Both make the scripting operations & command line interface easier by saving time for users.

People Also Ask

What is the difference between piping and redirection in bash?
Piping is used to pass the output of one command as input to another command, allowing you to chain commands together, while redirection is used to control the input and output sources and destinations for a single command.
What is redirection and piping in Linux?
Redirection is a process that allows you to change the default input source or output destination of a command. And, Bash piping is a powerful process that allows you to connect multiple commands in Linux and other Unix-like shell environments.

What is the key difference between a redirect (>) and piping to the tee command?
The key difference here is that while redirection (‘>’) only sends the output of a file and doesn’t display it on the terminal, piping to ‘tee’ allows you to see the output on the terminal as well as capture it in a file.

How do you use pipe and redirection?
Pipe & redirection are powerful concepts for managing input and output streams of commands. The pipe allows you to chain multiple commands together by passing one command’s output as input for another command. And, redirection controls where a command’s output goes and where it takes its input from.

Related Articles


<< Go Back to Bash Scripting Tutorial

Rate this post
Monira Akter Munny

Hello!! This is Monira Akter Munny. I'm a Linux content developer executive here, at SOFTEKO company. I have completed my B.Sc. in Engineering from Rajshahi University of Engineering & Technology in the Electrical & Electronics department. I'm more of an online gaming person who also loves to read blogs & write. As an open-minded person ready to learn & adapt to new territory, I'm always excited to explore the Linux world & share it with you! Read Full Bio

Leave a Comment