Bash Redirection

In Bash, the fundamental method for manipulating input and output is through redirection. Redirection allows you to alter the source or destination of a program’s file descriptors. It is a process of manipulating the input and output of commands in Bash scripting. From this writing, you will learn the basics, types & order of Bash redirection.

What is Redirection?

Redirection is a process that allows you to change the default input source or output destination of a command. With redirection, you can redirect output to a file instead of displaying it on the terminal, or instruct an application to read input from a file rather than the keyboard. Redirections are executed by the Bash shell before running the command to which the redirections are applied.

Standard Streams

Standard streams are redirected to files or from files in Linux. They are basically the ways of exchanging data between the programs and their environments. There are three types of standard streams, standard input, output & error. Through them, you get to control your script, like, from where it will read input, where its output will be destined, and how it handles error messages.

Now, in Linux, everything is treated as a file, and input and output are no exception. The OS assigns identifiers called file descriptors to represent these files. Each process can have a maximum of nine open file descriptors. In the bash shell, the first three descriptors are reserved with specific IDs for these standard streams. They are,

0 (represents STDIN)
1 (represents STDOUT)
2 (represents STDERR)

Check out the following graphical presentation of STDIN, STDOUT, and STDERR with a more detailed description for a clearer concept:

1. Standard Input (stdin)

Bash provides a way to interactively read input from the user or another command using standard input which allows scripts and programs to accept data directly from the keyboard or other input sources. You can read from stdin using the ‘read’ command, which allows you to assign input to variables for further processing in the script.

2. Standard Output (stdout)

Bash allows you to display output using standard output. By default, the output is displayed on the terminal. But you can also redirect the output to a file using the ‘>’ symbol or append it to a file using the ‘>>’ symbol. Moreover, you can also pipe the output of one command to another using the ‘|’ symbol.

3. Standard Error (stderr)

In addition to STDOUT, bash also provides a separate stream called standard error for handling error messages & diagnostic output. By default, STDERR is displayed on the terminal along with STDOUT. You can redirect STDERR to a file also using the ‘2>’ or append using the ‘2>>’ symbol.

Bash Redirection Operators

Check out the following list of redirection operators normally used in performing redirection operations in Bash:

>: Redirects standard output (stdout) to a file. If the file already exists, its contents will be overwritten.
>>: Redirects stdout to a file but appends the output to the end of the file if it already exists.
<: Redirects standard input (stdin) to a command from a file. The command will read input from the specified file.
2>: Redirects standard error (stderr) to a file. Error messages produced by the command will be written to the specified file.
2>>: Redirects stderr to file, but appends the error messages to the end of the file if it already exists.
&> or >&: Redirects both stdout and stderr to the same location. This operator is equivalent to using both ‘>file’ and ‘2>file’.
>>& or &>>: Redirects both stdout & stderr to the same file, but appends the output to the end of the file if it already exists.
<<: Represents ‘Here Document’, which allows you to provide input to a command within the script without needing an external file.
<<<: Represents ‘Here String’, which allows you to provide a string as the input to a command.
>|: Forces redirection to a file, even if the noclobber option (prevents accidental overwriting of existing files) is set.
2>&-: Tells Bash to close or discard the stderr stream in a Bash shell.
|: Pipe connects the stdout of one command to the stdin of another command, creating a data processing pipeline.

Various Cases of Bash Redirection

In the following writing, I will discuss various ways, you can redirect stdin, stdout & stderr of commands for your bash scripts. So, check out these cases of redirection available in Linux:

You can view the Summarized Table of the command syntaxes that you can use for redirection purposes.

A. Bash Input Redirection

Redirecting input is a process where the standard input (stdin) of a command is changed from the keyboard to a file. By, using the ‘<’ symbol you can take input data from a file instead of typing it interactively. For example,

cat < file.txt

The input used for the ‘cat command’ here is redirected from the ‘file.txt’ file.

B. Bash Output Redirection

Here, the displaying of the standard output (stdout) of a command is changed from the terminal screen to a file or another command’s input. The ‘>’ symbol is used for this purpose. For example,

whoami > output.txt

The output of the ‘whoami command’ is redirected to the ‘output.txt’ file. Which I viewed using the ‘cat command’.

C. Appending Redirected Output

Normally, when you redirect a stdout to a file with the ‘>’ redirection operator, it overwrites the file. You can also append the stdout to the existing file contents while redirecting by using the ‘>>’ redirection operators. For example,

whoami >> output.txt

Here, I used the append operator (>>), and as a result, instead of overwriting it, the command output is appended to the ‘output.txt’ file.

D. Bash Standard Error Redirection

The standard error (stderr) is a way to control where error messages, generated by commands are sent. You can redirect stderr to a file, discard it, or merge it with the standard output. You can use the redirection operator “2>” to redirect the standard error of a command to a file, “2>>” to append, or “2> /dev/null” to discard it. For example,

whoami -l 2> output.txt

From the image, you can see I used the wrong option for the command ‘whoami’. As a result, the command output shows an error message. However, it was not displayed on the screen, as was redirected to the ‘output.txt’ file. However, I viewed it later using the ‘cat command’.

Now, let’s discuss two special cases of stderr:

Bash Echo to Stderr

Here, the output of the echo command will be redirected to the stderr. And for that, the operator ‘>&2’ is used. Where ‘2’ represents the file descriptor for stderr. So, ‘>&2’ redirects the command output to the standard error stream. For example,

echo “This is an error message!” >&2

Bash Stderr to Stdout

In this case, the stderr will be redirected to the stdout. For that, you can use the operator ‘2>&1’. This will cause both stderr and stdout to be combined and directed to the same location. For example,

command > file 2>&1

E. Redirecting Standard Output and Standard Error

In Linux, every process has two default output streams, stdout (represented by file descriptor 1) & standard error or stderr (represented by file descriptor 2). You can redirect both of them to the same file using the symbol ‘&> or ‘>&’ or ‘2>&1’. For example,

ls /nonexistent_directory &> log.txt

The output & output error messages both are redirected to the ‘log.txt’ file without being displayed on the screen. Later I viewed the ‘log.txt’ file using the cat command.

F. Appending Standard Output and Standard Error

In the same way as appending stdout, you can append both stdout & stderr at the same time to the same file too. For that, use the append symbol ‘>>’. For example,

ls /nonexistent_directory >> output.txt 2>&1

Here, I used the append operator (>>) to append both stdout & stderr, and as a result, instead of overwriting it, the command output with the error message is appended to the ‘log.txt’ file. And later I viewed the log.txt file using the cat command.

Summarized Table for Redirection Syntaxes in Bash

In the following table, I tried to summarize all the cases you may need while performing redirection with their generalized command syntaxes:

Cases	Command Syntaxes
Redirect input	command < file.txt
Redirect Output	command > output.txt
Append Output	command >> output.txt
Redirect Standard Error	command 2> error.txt
Append Standard Error	command 2>> error.txt
Redirect stdout & stderr to the same file	command &> file.txt
Redirect stdout & stderr to different files	command >output.txt 2>error.txt
Append stdout & stderr to the same file	command &>> output.txt
Suppressing stderr	command 2>&-
Redirect stderr to stdout	command 1>&2

Order of Redirection: “>file 2>&1” vs. “2>&1>file”

The order of redirection in Bash commands matters, and it can result in different outcomes depending on where you place them. They are set up from left to right. For a clear understanding see the differences between the following command syntaxes:

command > file 2>&1

Here, the stdout is redirected to the ‘file’, and then the stderr is redirected to the same location as the stdout. This means both stdout and stderr will be sent to the file named ‘file’.

command 2>&1 > file

In this case, the stderr is redirected to the same location as the stdout, which is initially the terminal. Then, the stdout is redirected to the file ‘file’. This means stdout will go to the file ‘file’, but stderr will still go to the terminal because stderr’s redirection happened before stdout’s redirection.

Few Tips on Effective Redirection

The Bash shell maintains a flexible stance on what it defines as a valid redirection. So remember these points while performing redirection operations to avoid unnecessary errors:

Maintain tight grouping of redirections. That means, avoiding adding spaces within redirection syntax except within quotes when necessary (e.g. for filenames with spaces).
Use whitespace to separate redirections visually, creating chunks without unnecessary spaces.
Place a space between each redirection and between the argument list and the initial redirection.
Position redirections at the command’s end, following all arguments.
Do not start a command with a redirection or insert one in the middle of arguments.

Conclusion

To sum up, now that you know, using Bash redirection you can redirect input and output streams to or from files to perform complex operations and automate tasks. You can get an idea of how useful it can be while you are writing Bash scripts. In this writing, I tried to discuss the overall redirection basics you need to know. Hope it helps!