An Overview of Shell Expansion in Bash

Expansion is a useful technique for expanding certain types of characters or symbols in Bash. These characters, symbols or syntax are not just literal and have different meanings to Bash shell. These allow users to create a large number of tokens just by writing a few letters. In this article, I will discuss different types of shell expansion in Bash and their implementation.

Types of Shell Expansion in Bash

There are different types of shell expansion. The main types of expansions are:

  1. Brace Expansion
  2. Tilde Expansion
  3. Parameter Expansion
  4. Command Substitution
  5. Arithmetic Expansion
  6. Word Spiting
  7. Filename Expansion or Glob Expansion
  8. Process Substitution.

All these expansions combined are known as shell expansion. I will discuss each of these briefly.

Before that, you can run man bash and search for the text ^EXPANSION to see the manual page.man page of Bash shell expansionThe man page has everything you need to know about bash expansion.Expansion details in man page

1. Brace Expansion in Bash Shell

Brace expansion is a Bash expansion that allows users to generate arbitrary strings. The list that needs to be expanded must be put with curly “{}” braces. For example, the comma-separated list inside the curly braces of the following command expands to three different words.

echo {Ubuntu,Redhat,Fedora}

One can even generate a sequence of numbers with increments or decrements as well. For example, let’s create a sequence from 1 to 15 with an increment of 2.

echo {1..15..2}

Brace expansion of range

2. Bash Tilde Expansion

The tilde(~) is a character that usually refers to the user’s home directory or value of the environment variable “HOME”. This shortcut is quite useful when changing directories from the command line.Tilde expansion in BashHere, echo ~ expands the tilde sign and returns the value of the “HOME” variable. Let’s change the HOME variable to /usr using the export command.

export HOME=/usr

Setting new value for HOME variableThis set a new value to the HOME variable. Now, ~ refers to the new value. You can check it by cd ~. It will change the current directory to /usr. You can further check it using the pwd command.

Checking tilde expansion and new value of HOME variableAs you can see, the current directory is /usr, exactly what is currently set in the HOME variable.

HOME variable may not be set in the shell. So, tilde (~) can’t refer to the HOME variable. In that case, it refers to the home directory of the current user by default.

3. Parameter and Variable Expansion in Bash

Parameter expansion is a technique to expand a particular parameter or value of a variable. “$” symbol is used for parameter expansion. Typically, “${parameter_name}” expands a parameter with that particular name.

user=Jhon
echo ${user}

Parameter expansion in BashOne can use parameter expansion to extract or modify substrings and access different attributes of a parameter. For example- The simple syntax “${#parameter_name}” can find the length of a parameter.Length of a parameter using parameter expansion in Bash

Curly braces are not mandatory for parameter expansion. However, it is useful to avoid ambiguity.

4. Command Substitution in Bash

Command substitution is a technique of replacing a command name with the output of the command itself. It is useful for capturing the output of a command in a variable or passing the output as an argument of another command. Command substitution occurs whenever a command name is enclosed by parenthesis after the dollar sign ($).

$(command_name)

Backtick could be an alternative to the above syntax.

`command name`

Command substitution in BashIn the image, the date command is substituted using both the dollar sign and the backtick substitution.

Backtick is the older version of command substitution. Nested substitution using backtick can be confusing. In a nested substitution, one can escape inner backticks with backslashes.

5. Bash Arithmetic Expansion

Arithmetic Expansion in Bash is a way to expand and evaluate an arithmetic expression. The syntax of arithmetic expansion is $((expression)).

Inside the double parentheses, one can put a mathematical expression with numbers, variables or operators etc. The shell will evaluate the expression and replace the “$((expression))” construct with the result of the calculation. Let’s calculate the sum of 4 and 5 in the terminal.

echo $((4 + 5))

Arithmetic expansion in BashAs you can see the command performs the addition of 4 and 5 using arithmetic expansion. A few important properties of arithmetic expansion are-

  • Arithmetic expansion may be nested. For example- $((2+$((4+6))))
  • It follows the PEMDAS rule of precedence.
  • Sub-expressions or nested expressions may override the precedence rules.
  • An invalid expression prints a message indicating failure to the standard error and no substitution occurs.
  • Arithmetic expansion can’t handle floating point numbers. The bc command is useful for this purpose.

“$[ EXPRESSION ]”  can be an alternative to arithmetic expansion. Let’s perform the multiplication of 24*7 using this construction.Alternative of arithmeric expansion in Bash

6. Word Splitting in Bash

For word splitting, the shell examines the results of parameter expansion, command substitution, and arithmetic expansion that did not occur within double quotes. It treats each character of the IFS (Internal Field Separator) variable as a delimiter and splits the result of other expansions into words. IFS follows a few rules while splitting into words.

  1. If IFS is unset or set to the default value of <space><tab><newline>, leading and trailing whitespace is ignored.
  2. For non-default IFS values, leading and trailing whitespace are ignored if they match any character in IFS.
  3. Any character in IFS that is not whitespace, along with adjacent whitespace characters, delimits a field.
  4. Sequences of IFS whitespace characters are also treated as delimiters.

Take the example of “Hello World”. If IFS is unset or set to the default, it is split into two words: “Hello” and “World”.

Now, let’s say IFS is set to “,”. The adjacent whitespace with the delimiter character split “Ubuntu, Redhat, Fedora” into “Ubuntu”, “Redhat” and “Fedora”

If no expansion occurs, no splitting will be performed.

7. Filename Expansion in Bash Shell

The filename expansion or glob expansion is a way to match filenames in Bash. Whenever, Bash finds the characters ‘*’, ‘?’, and ‘[’ unquoted it treats the characters as a pattern and replaces them with filenames matching the pattern.

The “?” (question mark) in the context of glob expansion matches with a single character. For example, “user?.txt” can match with user1.txt or users.txt or any other character in place of the question mark. Look at the script below to see the execution.

#!/bin/bash
echo "Matching text files:"

for file in user?.txt; do
    echo "$file"
done

The program uses a for loop to iterate through all the files in the current directory that match the glob pattern “user?.txt”. Here, the “?” character is used to match a single character. For each matching file, the script echoes its name in the terminal.Pathname expansion in BashUpon execution “user?.txt” matches with the filename “users.txt” where the question mark matches the character “s”.

8. Process Substitution in Bash

Process substitution is a technique that allows users to use the output of a command or process as input for another command. It treats the output of a command as a file.

It takes two form-

command <(list) syntax creates a named pipe and the list passed as an argument should be read to obtain the output of command. Consider a situation where you want to compare the contents of two files without creating any temporary files.

grep 'Error' logfile | tee >(cat > errors.txt)

Process substitution in BashHere, the outputs of cat file1.txt and cat file2.txt are passed as the inputs of the diff command using command substitution.

command >(list) syntax is for writing to the file that will provide input for the list in this case. Suppose you want to find entries containing the word “Error” from a log file. You can use the following command if you want to save the error entries to a separate file simultaneously displaying them on the screen.

grep 'Error' logfile | tee >(cat > errors.txt)

Benefits of Shell Expansion in Bash

There are countless benefits of using shell expansion. A few of them are discussed below-

  1. Easy File Management: Pathname expansion (globbing) simplifies file and directory operations. Wildcards help to match and find files efficiently. Tilde expansion is the fastest way to move to the directory. Brace expansion has the potential to create or match a series of files or directories.
  2. Quick Manipulation of Shell parameters: Parameter expansion is helpful to manipulate variables and extract substrings from strings easily. One can easily convert a parameter to uppercase or lowercase by placing a caret sign at the end of the parameter name. For example, “${user^}” is used to capitalize the first letter of the “user” variable.
  3. Evaluation of Math Expression: Arithmetic expansion allows to perform mathematical calculations within scripts. This is quite useful when conditional statements need to evaluate math expressions.
  4. Capturing command output: Command substitution offers to capture the output of a command. It helps to write the output to a file, use the output as the argument of another command or save the output to a variable.

Potential Pitfalls of Bash Expansion

Apart from the benefits, sometimes users find it difficult to use expansion in scripting. Following are the issues that often arise while using expansion in the Bash shell.

  • Syntax Complexity: All Bash expansions are heavy with syntax. The syntax error is the

main reason for the unexpected behavior of Bash expansion. Some syntax constructions are not even rigid. So users often get confused and end up with the wrong expansion.

  • Complex Nesting: Excessive or deeply nested expansions can make commands

unreadable, especially in case of command substitution. For example, the command: `echo `date` | wc -w` is really tough to grasp due to its nested nature.

  • Unintended Expansion: Variables and commands inside double quotes or backticks

are expanded whereas inside single quotes it doesn’t. This can lead to unintentional expansions if not properly escaped or quoted.

  • Portability: Some Bash features and expansions may not be portable across different

Shells like- Bash to zsh or even within different versions of Bash. Hence, users need to be aware of the version of Bash. To give an example process substitution is not available in all shells. It is a feature in Bash shell and a few other shells like ZSH but is not available in POSIX compliant.

Advanced Techniques of Bash Shell Expansion

Bash expansion is full of tricks and techniques. Let’s explore a couple of advanced techniques of expansion in Bash.

Negative Increment to Create a Sequence Using Brace Expansion

To create a sequence with a negative increment use the negative increment value while expanding a range. For creating a sequence from 15 to 1 with a decrement of 2 use the following command.

echo {15..1..-2}

Brace expansion with negative increment

Indirect Referencing in Parameter Expansion

In parameter expansion, “${!parameter_name}” refers to a level of indirection. This means Bash first expands the name after the exclamation sign. Then the value of the first expansion is used as a new parameter. It then expands the new parameter, rather than using the original parameter’s expansion.

Conclusion

In conclusion, expansion in Bash is a pretty big topic. It includes lots of different concepts. This article tries to give you an overview of expansion in Bash. I would recommend visiting individual articles on each type of expansion for more in-depth information.

People Also Ask

What is the first expansion in Bash?

Brace expansion is the first expansion in Bash. The overall order of expansions is brace expansion, tilde expansion, parameter and variable expansion, arithmetic expansion, command substitution (left-to-right), word splitting and filename expansion.

How to perform Parameter Expansion on arrays in Bash shell?

To perform parameter expansion on an array variable use the syntax ‘${array_name[@]}’. If you have an array like- arr1=(‘null’ 0 10), then ‘${arr1[@]}’ expands all the elements of that array.

Which expansions increase the number of words?

Brace expansion, word splitting, and filename expansion can increase the number of words. On the contrary, other expansions expand a single word to a single word. However, there are a few exceptions. For example, the special parameters ‘$@’ and ‘$*’ as well as array expansion using ‘${array[@]}’ and ‘${array[*]}’.

When does process substitution work in shell expansion?

Based on the availability, process substitution works in shell expansion simultaneously with parameter expansion, command substitution, and arithmetic expansion.

How to perform the uppercase operation in parameter expansion?

To convert the value of the parameter to uppercase use the caret ^ sign at the end of the parameter. To be specific, ‘${user^}’ capitalizes the first letter of the ‘user’ variable. And ‘${user^^}’ capitalizes the whole ‘user’ variable.

Related Articles


<< Go Back to Bash Scripting Tutorial 

5/5 - (1 vote)
LINUX
FUNDAMENTALS
A Complete Guide for Beginners Enroll Course Now
icon linux
Md Zahidul Islam Laku

Hey, I'm Zahidul Islam Laku currently working as a Linux Content Developer Executive at SOFTEKO. I completed my graduation from Bangladesh University of Engineering and Technology (BUET). I write articles on a variety of tech topics including Linux. Learning and writing on Linux is nothing but fun as it gives me more power on my machine. What can be more efficient than interacting with the Operating System without Graphical User Interface! Read Full Bio

Leave a Comment