FUNDAMENTALS A Complete Guide for Beginners
Expansion is a useful technique for expanding certain types of characters or symbols in Bash. These characters, symbols or syntax are not just literal and have different meanings to Bash shell. These allow users to create a large number of tokens just by writing a few letters. In this article, I will discuss different types of shell expansion in Bash and their implementation.
Types of Shell Expansion in Bash
There are different types of shell expansion. The main types of expansions are:
- Brace Expansion
- Tilde Expansion
- Parameter Expansion
- Command Substitution
- Arithmetic Expansion
- Word Spiting
- Filename Expansion or Glob Expansion
- Process Substitution.
All these expansions combined are known as shell expansion. I will discuss each of these briefly.
Before that, you can run man bash and search for the text ^EXPANSION to see the manual page.
The man page has everything you need to know about bash expansion.
1. Brace Expansion in Bash Shell
Brace expansion is a Bash expansion that allows users to generate arbitrary strings. The list that needs to be expanded must be put with curly “{}” braces. For example, the comma-separated list inside the curly braces of the following command expands to three different words.
echo {Ubuntu,Redhat,Fedora}One can even generate a sequence of numbers with increments or decrements as well. For example, let’s create a sequence from 1 to 15 with an increment of 2.
echo {1..15..2}
2. Bash Tilde Expansion
The tilde(~) is a character that usually refers to the user’s home directory or value of the environment variable “HOME”. This shortcut is quite useful when changing directories from the command line.
Here, echo ~ expands the tilde sign and returns the value of the “HOME” variable. Let’s change the HOME variable to /usr using the export command.
export HOME=/usr
This set a new value to the HOME variable. Now, ~ refers to the new value. You can check it by cd ~. It will change the current directory to /usr. You can further check it using the pwd command.
As you can see, the current directory is /usr, exactly what is currently set in the HOME variable.
3. Parameter and Variable Expansion in Bash
Parameter expansion is a technique to expand a particular parameter or value of a variable. “$” symbol is used for parameter expansion. Typically, “${parameter_name}” expands a parameter with that particular name.
user=Jhon
echo ${user}
One can use parameter expansion to extract or modify substrings and access different attributes of a parameter. For example- The simple syntax “${#parameter_name}” can find the length of a parameter.
4. Command Substitution in Bash
Command substitution is a technique of replacing a command name with the output of the command itself. It is useful for capturing the output of a command in a variable or passing the output as an argument of another command. Command substitution occurs whenever a command name is enclosed by parenthesis after the dollar sign ($).
$(command_name)
Backtick could be an alternative to the above syntax.
`command name`
In the image, the date command is substituted using both the dollar sign and the backtick substitution.
5. Bash Arithmetic Expansion
Arithmetic Expansion in Bash is a way to expand and evaluate an arithmetic expression. The syntax of arithmetic expansion is $((expression)).
Inside the double parentheses, one can put a mathematical expression with numbers, variables or operators etc. The shell will evaluate the expression and replace the “$((expression))” construct with the result of the calculation. Let’s calculate the sum of 4 and 5 in the terminal.
echo $((4 + 5))
As you can see the command performs the addition of 4 and 5 using arithmetic expansion. A few important properties of arithmetic expansion are-
- Arithmetic expansion may be nested. For example- $((2+$((4+6))))
- It follows the PEMDAS rule of precedence.
- Sub-expressions or nested expressions may override the precedence rules.
- An invalid expression prints a message indicating failure to the standard error and no substitution occurs.
- Arithmetic expansion can’t handle floating point numbers. The bc command is useful for this purpose.
“$[ EXPRESSION ]” can be an alternative to arithmetic expansion. Let’s perform the multiplication of 24*7 using this construction.
6. Word Splitting in Bash
For word splitting, the shell examines the results of parameter expansion, command substitution, and arithmetic expansion that did not occur within double quotes. It treats each character of the IFS (Internal Field Separator) variable as a delimiter and splits the result of other expansions into words. IFS follows a few rules while splitting into words.
- If IFS is unset or set to the default value of <space><tab><newline>, leading and trailing whitespace is ignored.
- For non-default IFS values, leading and trailing whitespace are ignored if they match any character in IFS.
- Any character in IFS that is not whitespace, along with adjacent whitespace characters, delimits a field.
- Sequences of IFS whitespace characters are also treated as delimiters.
Take the example of “Hello World”. If IFS is unset or set to the default, it is split into two words: “Hello” and “World”.
Now, let’s say IFS is set to “,”. The adjacent whitespace with the delimiter character split “Ubuntu, Redhat, Fedora” into “Ubuntu”, “Redhat” and “Fedora”
If no expansion occurs, no splitting will be performed.
7. Filename Expansion in Bash Shell
The filename expansion or glob expansion is a way to match filenames in Bash. Whenever, Bash finds the characters ‘*’, ‘?’, and ‘[’ unquoted it treats the characters as a pattern and replaces them with filenames matching the pattern.
The “?” (question mark) in the context of glob expansion matches with a single character. For example, “user?.txt” can match with user1.txt or users.txt or any other character in place of the question mark. Look at the script below to see the execution.
#!/bin/bash
echo "Matching text files:"
for file in user?.txt; do
echo "$file"
doneThe program uses a for loop to iterate through all the files in the current directory that match the glob pattern “user?.txt”. Here, the “?” character is used to match a single character. For each matching file, the script echoes its name in the terminal.
Upon execution “user?.txt” matches with the filename “users.txt” where the question mark matches the character “s”.
8. Process Substitution in Bash
Process substitution is a technique that allows users to use the output of a command or process as input for another command. It treats the output of a command as a file.
It takes two form-
command <(list) syntax creates a named pipe and the list passed as an argument should be read to obtain the output of command. Consider a situation where you want to compare the contents of two files without creating any temporary files.
grep 'Error' logfile | tee >(cat > errors.txt)
Here, the outputs of cat file1.txt and cat file2.txt are passed as the inputs of the diff command using command substitution.
command >(list) syntax is for writing to the file that will provide input for the list in this case. Suppose you want to find entries containing the word “Error” from a log file. You can use the following command if you want to save the error entries to a separate file simultaneously displaying them on the screen.
grep 'Error' logfile | tee >(cat > errors.txt)
Benefits of Shell Expansion in Bash
There are countless benefits of using shell expansion. A few of them are discussed below-
- Easy File Management: Pathname expansion (globbing) simplifies file and directory operations. Wildcards help to match and find files efficiently. Tilde expansion is the fastest way to move to the directory. Brace expansion has the potential to create or match a series of files or directories.
- Quick Manipulation of Shell parameters: Parameter expansion is helpful to manipulate variables and extract substrings from strings easily. One can easily convert a parameter to uppercase or lowercase by placing a caret sign at the end of the parameter name. For example, “${user^}” is used to capitalize the first letter of the “user” variable.
- Evaluation of Math Expression: Arithmetic expansion allows to perform mathematical calculations within scripts. This is quite useful when conditional statements need to evaluate math expressions.
- Capturing command output: Command substitution offers to capture the output of a command. It helps to write the output to a file, use the output as the argument of another command or save the output to a variable.
Potential Pitfalls of Bash Expansion
Apart from the benefits, sometimes users find it difficult to use expansion in scripting. Following are the issues that often arise while using expansion in the Bash shell.
- Syntax Complexity: All Bash expansions are heavy with syntax. The syntax error is the
main reason for the unexpected behavior of Bash expansion. Some syntax constructions are not even rigid. So users often get confused and end up with the wrong expansion.
- Complex Nesting: Excessive or deeply nested expansions can make commands
unreadable, especially in case of command substitution. For example, the command: `echo `date` | wc -w` is really tough to grasp due to its nested nature.
- Unintended Expansion: Variables and commands inside double quotes or backticks
are expanded whereas inside single quotes it doesn’t. This can lead to unintentional expansions if not properly escaped or quoted.
- Portability: Some Bash features and expansions may not be portable across different
Shells like- Bash to zsh or even within different versions of Bash. Hence, users need to be aware of the version of Bash. To give an example process substitution is not available in all shells. It is a feature in Bash shell and a few other shells like ZSH but is not available in POSIX compliant.
Advanced Techniques of Bash Shell Expansion
Bash expansion is full of tricks and techniques. Let’s explore a couple of advanced techniques of expansion in Bash.
Negative Increment to Create a Sequence Using Brace Expansion
To create a sequence with a negative increment use the negative increment value while expanding a range. For creating a sequence from 15 to 1 with a decrement of 2 use the following command.
echo {15..1..-2}
Indirect Referencing in Parameter Expansion
In parameter expansion, “${!parameter_name}” refers to a level of indirection. This means Bash first expands the name after the exclamation sign. Then the value of the first expansion is used as a new parameter. It then expands the new parameter, rather than using the original parameter’s expansion.
Conclusion
In conclusion, expansion in Bash is a pretty big topic. It includes lots of different concepts. This article tries to give you an overview of expansion in Bash. I would recommend visiting individual articles on each type of expansion for more in-depth information.
People Also Ask
What is the first expansion in Bash?
Brace expansion is the first expansion in Bash. The overall order of expansions is brace expansion, tilde expansion, parameter and variable expansion, arithmetic expansion, command substitution (left-to-right), word splitting and filename expansion.
How to perform Parameter Expansion on arrays in Bash shell?
To perform parameter expansion on an array variable use the syntax ‘${array_name[@]}’. If you have an array like- arr1=(‘null’ 0 10), then ‘${arr1[@]}’ expands all the elements of that array.
Which expansions increase the number of words?
Brace expansion, word splitting, and filename expansion can increase the number of words. On the contrary, other expansions expand a single word to a single word. However, there are a few exceptions. For example, the special parameters ‘$@’ and ‘$*’ as well as array expansion using ‘${array[@]}’ and ‘${array[*]}’.
When does process substitution work in shell expansion?
Based on the availability, process substitution works in shell expansion simultaneously with parameter expansion, command substitution, and arithmetic expansion.
How to perform the uppercase operation in parameter expansion?
To convert the value of the parameter to uppercase use the caret ^ sign at the end of the parameter. To be specific, ‘${user^}’ capitalizes the first letter of the ‘user’ variable. And ‘${user^^}’ capitalizes the whole ‘user’ variable.
Related Articles
- How to Use Brace Expansion in Bash Scripting [3 Cases]
- How to Use Tilde Expansion in Bash [4 Practical Cases]
- Parameter Expansion in Bash [3 Main Types]
- Arithmetic Expansion in Bash [3 Practical Applications]
- An Exhaustive Guide to Bash History Expansion
- What is Array Expansion in Bash [4 Useful Applications]
- Glob Expansion in Bash
<< Go Back to Bash Scripting Tutorial
