FUNDAMENTALS A Complete Guide for Beginners
Glob expansion is one of the elegant techniques in Bash to find files or match pathnames. There are a few special characters to execute the technique efficiently such as “*”, “?” and “[” etc. In this article, I will talk about the use of those characters as well as various aspects of glob expansion in Bash.
What is Glob Expansion in Bash?
Glob expansion, also known as globbing, is a way to match filename or pathname in the Linux operating system. It allows users to use wildcard characters to match a desired pattern while searching for filenames or pathnames. It is a quick and efficient technique to find files and navigate directories. The most common wildcards used for globbing are “*”, “?”, “[”. Whenever Bash finds these characters unquoted, it tries to replace those with filenames matching the pattern defined by the wildcards. For example, “file.???” can match a filename like “file.txt” or “file.png” as “?” wildcard is used for single character matching.
Glob Expansion Using Different Wildcards in Bash
There are a couple of wildcard characters widely used for globbing in Bash. Those wildcards may look similar to regular expressions. However, these are inherently different from regular expression. The list below contains a few wildcard characters frequently used for globbing.
- * (Asterisk)
- ? (Question Mark)
- [ ] (Square Bracket)
- ! (Exclamation Mark)
- | (pipe)
1. Glob Expansion of Matching Any Character Using “*” (Asterisk)
Asterisk “*” is a powerful and widely used wildcard for globbing. It can match any length of characters including none. For instance, “*.txt” can match “file.txt”, “f.txt” or just “.txt”. Check the below script.
#!/bin/bash
echo "Matching text files:"
for file in *.txt; do
echo "$file"
doneThe script employs a for loop to iterate through all the files in the current directory that match the glob pattern “*.txt”. For each matching file, the script echoes its name in the terminal.
When I execute the program in my terminal, four files match the pattern. If you closely look at those, each of the files has a “.txt” extension. From the matching filenames it is evident that an asterisk sign matches with strings of various lengths.
2. Glob Expansion of Matching a Single Character Using the “?” Sign
The “?” (question mark) is a glob expansion that matches with a single character in Bash globbing. For example, “user?.txt” can match with “user1.txt” or “users.txt” or any other character in place of the question mark. Look at the script below to see the execution.
#!/bin/bash
echo "Matching text files:"
for file in user?.txt; do
echo "$file"
doneThe program uses a for loop to iterate through all the files in the current directory that match the glob pattern “user?.txt”. Here, the “?” wildcard is used to match a single character. For each matching file, the script echoes its name in the terminal.
While running the program in the terminal, the “user?.txt” matches with the filename “users.txt” where the question mark represents the single character “s”.
3. Glob Expansion Using Square Bracket “[ ]” in Bash
A [] (square bracket) is used to match a single character within the defined set of characters or ranges. For instance, the expression [a-c] will match any one of the characters a, b or c. However, it will not match the sequence ab because it’s designed to match a single character. Additionally, it will not match the character d since it is not included in the expression defined within the square brackets. Let’s see how to use it in a script.
#!/bin/bash
echo "Matching Directories:"
for dir in [p-xP-X]*; do
if [ -d "$dir" ]; then
ls -d "$dir"
fi
doneThe program displays directories within the current folder that have names starting with a letter in the range from “p” to “x” (both lowercase and uppercase). It iterates through the contents of the current folder, matching directory names that satisfy the pattern [p-xP-X]* using a for loop. For each matching, it checks if it is a directory using the conditional statement [ -d "$dir" ]. Then ls command with the “-d” option displays the directory’s name.
Upon execution, the program successfully prints directory names that start with characters falling within the range of “p” to “x” both in uppercase and lowercase.
Here’s the ranges that can be used within square brackets for matching glob patterns.
| Range | Explanation | Example | Sample Output |
|---|---|---|---|
| [:digit:] | Match any digit | file[0-9].txt | file1.txt or file5.txt |
| [:alpha:] | Match any uppercase or lowercase letters. | file[A-Z].txt | filea.txt or fileA.txt |
| [:alnum:] | Match any alphanumeric characters | [a-zA-z0-9] | file1.txt or fileA.txt or filea.txt |
Negating a Pattern in Glob Expansion
Negating a pattern means it can match anything except what is specified in the pattern. The caret (^) sign and exclamation mark are often used to negate a glob pattern.
Negating a Range Using Caret “^” in Glob Expansion
The caret (^) sign is used to negate a range of pattern. For example, [ab] matches either the character a or b while [^ab] will match any character except a and b. Here’s how to use it:
!/bin/bash
: '
List files or directories that
do not start with c-z or C-Z'
for file in [^c-zA-Z]*.sh; do
echo "$file"
doneThis script uses the pattern [^c-zA-Z]*.sh which means any file not starting with c-z and A-Z.
The image shows that the program finds the files of the current directory that do not start with any character from c to z (lowercase) or any character from A to Z (uppercase).
Negating a Pattern Using Exclamation “!” in Glob Expansion
One can use an exclamation mark as well to negate a pattern. To do this effectively, it’s required to enable extended globbing using shopt -s extglob command.
#!/bin/bash
shopt -s extglob # Enable extended globbing
# Files except those ending with .txt or .sh
for file in !(*.sh|*.txt); do
if [ -f "$file" ]; then
echo "$file"
doneThis script with extended globbing enabled, lists files in the current directory that do not end with either “.txt” or “.sh“. It achieves this by using a for loop with !(*.sh|*.txt) to match files with names that don’t have these extensions. The script then checks if each matched item is a file using the conditional statement [ -f "$file" ] before echoing their names.
The image shows that the script displays the files of the current directory that don’t have “.txt” or “.sh” extension.
Brace Expansion for Matching with Multiple Entities
Anything put inside unquoted curly braces is expanded and treated differently to the Bash shell. This Brace expansion is one of the shell expansion techniques in Bash. Globbing with brace expansion is so powerful. It can be used for matching files with different extensions at a time. Here’s an example:
#!/bin/bash
# Files with .txt or .png extension
ls -lh *.{txt,png}This Bash script lists and displays detailed information about files with either with a “.txt” or “.png” extension in the current directory. It uses curly braces or the brace expansion to match multiple entities at a time.
The image shows that the program lists multiple files with a “.txt” extension. But It doesn’t find any files with a “.png” extension.
Tab Completion to Match Glob Pattern
Tab completion is the fastest way to complete file names in your current directory. The TAB key can find and match with the filename when a part of it is written.
Let’s say I have a file called “find_replace.txt” in the current directory. After writing a substantial portion of the filename (say “find_”), pressing TAB expands the whole name.
After pressing TAB, I get the full filename as shown in the image below.
Recursive Globbing Using “globstar”
The globstar is an option that allows users to search not only within the current directory but also recursively through all its subdirectories. This means it looks for matches in the current directory and all its nested folders. For instance, the ls **/*.txt command should look for the pattern “*.txt” in the current directory and at all levels of subdirectories below the current directory. First, run this command and look at the output before enabling globstar.
As you can see, no file or directory matched with the pattern. Now, run the following command to enable globstar.
shopt -s globstarAgain, run ls **/*.txt command and see the differences.
This time, the command found multiple matches on my computer. If you examine “Documents/backup/cloud/file1.txt“, you’ll notice that the command searched for files and directories, including all their subdirectories, within the current directory.
Glob Expansion with Whitespaces
Glob expansion may not work properly for filenames or pathnames that contain whitespace. For example, a file named “my file.txt” can’t be listed using the ls command. It will end up with an error due to the whitespace.
The best way to deal with this problem is to set the Internal Field Separator (IFS) so that it can remove all the while whitespaces. The script shows how to use it:
IFS="$(printf '\n\t')" # Remove space.
# Correct glob use:
for file in *file.txt ; do
if [ -e $file ] ; then # Check whether the file exists.
ls -l $file
fi
doneThe script sets the Internal Field Separator (IFS) to remove spaces, tabs, and newlines. It then utilizes a for-loop to iterate through files in the current directory that match the pattern “*file.txt” For each file, it checks if the file exists using -e option in the condition. If the file exists, it runs the ls -l command to display detailed information about the file.
As shown in the image, the script lists “my file.txt” though it has a whitespace within its name.
How to Disable Glob Expansion
Disable globbing using set -f to prevent wildcard characters from being interpreted as part of glob expansion, . This is really useful when working with filenames or path names that contain wildcard characters. If it is required to enable it again use set +f.
However, when you use set -f to disable file globbing, variable settings are lost once the shell session ends. To address this, you can save the shell settings to a variable and later restore the settings using that variable.
#!/bin/bash
# Save the current shell options
shell_opts=$(set +o)
# Disable globbing
set -f
# Your code here without globbing
echo *.txt
# Restore shell options
eval "$saved_opts"The script begins by capturing the current shell options and storing them in the variable “shell_opts”. It then disables globbing, preventing wildcard characters “*” from being expanded. The “echo *.txt” command is used to illustrate this; it will print the literal string “*.txt” since globbing is turned off. Finally, it attempts to restore the original shell options using eval.
When you run the program in the terminal, it will print the literal “*.txt” as shown in the image.
However, when you run the command echo *.txt in the terminal the asterisk wildcard will be expanded if it finds any match like the following.
Conclusion
To conclude, glob expansion is frequently useful to the users of Bash shell. One can’t harness its full effectiveness without having a complete idea. I believe this article helped you to implement Glob Expansion at least in some obvious circumstances.
People Also Ask
Is the exclamation sign part of Globbing or Regular Expression in Bash?
Both. The exclamation works both in globbing and regular expression. In glob patterns, the exclamation mark is used to negate a pattern. For example, if you want to match all files except those with a ‘.txt’ extension, you can use ‘!(*.txt)‘.
Similarly, in the context of globbing the exclamation mark is used to negate a pattern in the case of regular expression as well. ‘[!ab]‘ will match any character except ‘a‘ or ‘b‘.
How can I turn off ‘globstar’ option in the Bash shell?
To turn off the globstar option in the Bash shell, use the shopt -u globstar command. This command turns off the globstar option, preventing recursive searching through subdirectories.
How to perform globbing within parameter expansion in Bash
To perform globbing within parameter expansion write the globbing pattern properly, make sure there is no syntax error. Suppose you want to use parameter expansion to access files that match a certain glob pattern like ‘*.txt‘. You can use $(…) to execute a command within parameter expansion and perform globbing.
matching_files=($(echo “*.txt”))
How does glob expansion with ‘[‘ differ from the ‘[[…]]’ syntax?
‘[‘ is primarily used for pattern matching or pathname expansion and the double square bracket is used to evaluate a condition, For example, ‘[aeiou]‘ matches any vowel. On the other hand, ‘[[ $var -gt 10 ]]‘ will check whether the ‘var‘ is greater than 10.
Related Articles
- How to Use Brace Expansion in Bash Scripting [3 Cases]
- How to Use Tilde Expansion in Bash [4 Practical Cases]
- Parameter Expansion in Bash [3 Main Types]
- Arithmetic Expansion in Bash [3 Practical Applications]
- An Exhaustive Guide to Bash History Expansion
- What is Array Expansion in Bash [4 Useful Applications]
<< Go Back to An Overview of Shell Expansion in Bash | Bash Scripting Tutorial
