Bash, a popular shell in Unix-like systems, provides two essential features for managing the flow of data between commands and files. These are redirection and piping. From this writing, you will learn about the basics of Bash redirection and piping, their differences & cases where you can apply them while Bash scripting.
Redirection in Bash
Redirection is a process that allows you to change the default input source or output destination of a command. With redirection, you can redirect output to a file instead of displaying it on the terminal, or instruct an application to read input from a file rather than the keyboard. You can redirect your used script commands in different ways whenever necessary. So, the following are the ways of redirection available in Linux:
A. Redirecting Input
Redirecting input is a process where the standard input (stdin) of a command is changed from the keyboard to a file. By, using the ‘<’ symbol you can take input data from a file instead of typing it interactively. For example,
cat < file.txtB. Redirecting Output
Here, the displaying of the standard output (stdout) of a command is changed from the terminal screen to a file or another command’s input. The ‘>’ symbol is used for this purpose. For example:
ls > output.txtC. Redirecting Standard Error
The standard error (stderr) is a way to control where error messages, generated by commands are sent. You can redirect stderr to a file, discard it, or merge it with the standard output. You can use the redirection operator “2>” to redirect the standard error of a command to a file, “2>>” to append, or “2> /dev/null” to discard it. For example,
whoami -l 2> output.txtD. Appending Redirected Output
Normally, when you redirect a stdout to a file with the ‘>’ redirection operator, it overwrites the file. You can also append the stdout to the existing file contents while redirecting by using the ‘>>’ redirection operators. For example,
ls >> output.txtE. Redirecting Standard Output and Standard Error
In Linux, every process has two default output streams, stdout (represented by file descriptor 1) & standard error or stderr (represented by file descriptor 2). You can redirect both of them using the symbol ‘&> or ‘>&’ or ‘2>&1’. For example,
ls /nonexistent_directory &> log.txtF. Appending Standard Output and Standard Error
In the same way as appending stdout, you can append both stdout & stderr at the same time to the same file too. For that, use the append symbol ‘>>’. For example,
ls >> output.txt 2>&1Piping in Bash
Bash piping is a powerful process that allows you to connect multiple commands in Linux and other Unix-like shell environments. In a more simple context, a pipe redirects the standard output from one command to another for processing. By using the pipe character ‘|’, you can connect one command’s output to another command’s input.
Pipes are unidirectional, flowing data from left to right through the pipeline.
The Command Syntax >
Command 1 | command 2 | … | Command NDifference Between Bash Redirection & Piping
Some users may find the operations of Bash redirection & piping quite similar. However, they have some core differences as they serve different purposes. Read out the following key points of their differences:
- A pipe (|) directs the output of one program to another program or utility. Whereas, redirection (‘>’, ‘>>’, ‘<’, etc) controls where the output goes or comes from.
- Pipe facilitates seamless data flow between commands in a sequence. And, redirection makes sure that output is redirected to a file (‘>’) or read from a file (‘<’).
- Piping is unidirectional, sending data from left to right whilst redirection handles input & output separately.
- Each serves a distinct purpose. While pipe does is collaboration, redirection emphasizes controlling inputs & outputs.
Let me simplify the two cases a bit more with the following example:
Program 1 > Program 2 vs Program 1 | Program 2
‘Program 1 > Program 2’
As soon as you execute it, your shell initiates the execution of the program labeled as ‘Program 1’. All output generated by ‘Program 1’ is stored within a file designated as ‘Program 2’ (please note, if ‘Program 2’ exists, its contents are overwritten).
‘Program 1 | Program 2’
On the other hand, if the intention is to forward the output from ‘Program 1’ to ‘Program 2’, the following approach can be used:
Program 1 > temp_file && Program 2 < temp_fileHere, first, your shell will launch ‘Program 1’, and save the output to a file name ‘temp_file’. After that, will initiate the execution of ‘Program 2’, treating the contents of ‘temp_files’ as if they were input from the keyboard.
However, this approach could be tedious. Therefore, the concept of pipes was introduced to simplify the process. So, ‘Program 1 | Program 2’ accomplishes the same objective as ‘Program 1 > temp_file && Program 2 < temp_file’.
A Sample Bash Script Example Combining Redirection and Piping
Now, let’s explore a practical example, where I will use both Bash redirection & piping processes in a single command line. To check yourself, copy the following Bash script on your nano editor & save the file with an appropriate name with the .sh extension.
Script (test.sh) >
#! /bin/bash
ls -l | grep ‘.txt’ > text_files
cat text_filesAfterward, the redirection operator ‘>’ redirects the standard output of the command sequence ‘ls -l | grep ‘.txt’’ to a file named ‘text_files’. If the file doesn’t exist, it will be created. If it exists, its contents will be overwritten. Finally, the cat command displays the contents of the file.
After that, use the following command to make the script executable:
chmod u+x test.sh- chmod: Changes the permission of files and directories.
- u+x: Argument with chmod command to add the executable permission for the user.
- test.sh: File which you want to make executable.
Finally, run the script by the following command:
./test.sh
The output of the script displays all the files with ‘.txt’ extension.
Conclusion
To sum up, Bash redirection and piping are vital Unix tools. Redirection controls I/O flow, while piping chains commands. Both make the scripting operations & command line interface easier by saving time for users.
People Also Ask
Related Articles
<< Go Back to Bash Scripting Tutorial
FUNDAMENTALS A Complete Guide for Beginners
