How to Remove Empty Lines in Bash? [5 Methods]

In Bash, Empty Lines refer to the lines that contain only whitespace characters or are completely blank. Empty Lines in text files, multiline strings, or scripts can sometimes clutter the content and make it harder to read or process. To remove empty lines in Bash, various approaches can be adopted. This article aims to present 5 methods for removing empty lines from text files or multiline strings in Bash.

You can follow the 5 methods to remove empty lines in Bash:

  1. Using the “sed” command: sed '/^[[:space:]]*$/d'
  2. Using the “grep” command: grep -v '^$'
  3. Using “awk” command: awk '!/^[[:space:]]*$/'
  4. Using the “tr” command: tr -s '\n'
  5. Using “perl”: perl -ne 'print if /\S/'

Let’s dive into the 5 methods to learn how to remove empty lines in Bash:

1. Using “sed” Command

The sed command is a powerful command-line utility for parsing and transforming text. You can use the syntax sed '/^[[:space:]]*$/d' filename” to remove the empty lines from a text file or multiline string using the sed command.

For example, let’s consider a text file with the following content:

#inputfile.txt
First Line

Second Line


Third Line


Fourth Line

To remove empty lines from a file, follow the script below:

#!/bin/bash
file="input.txt"

echo "Original file contents:"
cat "$file"

echo "Removing empty lines..."

# Use sed to delete empty lines
sed '/^[[:space:]]*$/d' "$file" > temp.txt

# Replace the original file with the temporary file
mv temp.txt "$file"

echo "Empty lines removed. Updated file contents:"
cat "$file"

EXPLANATION
This script removes empty lines from a file named “input.txt”. It utilizes sed with the /^[[:space:]]*$/d pattern to delete (d) lines that contain only whitespace characters. The syntax /^[[:space:]]*$/d, // holds the search string, ^ presents the start, $ presents the end, and d removes the matched string. Finally, the mv command replaces the original file with the temporary one.

Removing the empty lines from a text file using "sed" command.The output of the script shows the content of the text file before and after removing the empty lines.

Remove Leading Empty Lines Only

You can remove the leading blank lines only using the sed command using the expression sed '1,/\S/{/\S/!d}'. Follow the script below to remove the leading empty lines:

#!/bin/bash

# Multiline string with leading blank lines

multiline_string="


First Line
Second Line

Third Line"

#print the multiline string
echo "The Multiline String:"
echo "$multiline_string"

echo

# Remove leading blank lines using sed
clean_string=$(echo "$multiline_string" | sed '1,/\S/{/\S/!d}')

# Print the result
echo "After removing the leading empty lines:"
echo "$clean_string"

EXPLANATION
The script uses the sed command with the expression '1,/\S/{/\S/!d}' to remove the leading empty lines from the multiline string. Here, 1,/\S/ specifies a range from the first line (1) to the first line containing a non-whitespace character (/\S/). \S represents any non-whitespace character. {/\S/!d} is a condition that checks if the current line contains a non-whitespace character and deletes the line if the condition /\S/ is not satisfied. The script avoids all the blank lines except the leading one.

Removes the leading blank line using the sed commandThe output shows the multiline string before and after removing the leading blank lines from the multiline string using the sed command.

Remove Trailing Empty Lines Only

A trailing empty line refers to a line containing only whitespace characters or no characters at all, located at the end of a block of text or file. To remove the trailing empty lines using the sed command, follow the script below:

#!/bin/bash

# Multiline string with trailing blank lines
multiline_string="
First Line
Second Line
Third Line



"

#print the multiline string
echo "The Multiline String:"
echo "$multiline_string"
echo
# Remove trailing blank lines using sed
new_string=$(echo "$multiline_string" | sed -e :a -e '/^\n*$/{$d;N;ba' -e '}')

# Print the result
echo "After removing the leading empty lines:"
echo "$new_string"
EXPLANATION
In the script the command sed -e :a -e '/^\n*$/{$d;N;ba' -e '}' consists of multiple parts. Firstly, -e :a defines a label “a” for the sed command. Next, /^\n*$/ is a regular expression pattern that matches lines consisting of zero or more whitespace characters followed by a newline character, identifying empty lines. Within the curly braces, {$d;N;ba}, several commands are executed for lines matching the pattern.

Removes the trailing blank lines using the sed commandThe output shows the multiline string before and after removing the trailing empty lines using the sed command.

2. Using “grep” Command

The grep command is a versatile command-line tool for searching patterns in text. This command can be leveraged to remove the empty lines using the option v. The syntax is grep -v '^[[:space:]]*$' filename. Check the following script to delete empty lines using the “grep” command:

#!/bin/bash

#Define a multiline string
multiline_string="First Line

Second Line

Third Line


Fourth Line"

#Print the multiline string
echo "The Multiline String:"
echo "$multiline_string"
echo

#remove the empty lines
cleaned_string=$(echo "$multiline_string" | grep -v '^$')
multiline_string="$cleaned_string"

#print the cleaned string
echo "The multiline string after removing the empty lines"
echo "$multiline_string"

EXPLANATION
The command grep -v '^$' utilizes the grep tool with the -v option, instructing it to display lines that do not match a specified pattern. In this case, the pattern '^$' represents empty lines. The caret (^) anchors the pattern to the beginning of a line, while the dollar symbol ($) anchors the pattern to the end. More specifically, this expression defines the empty lines.

Bash remove the empty lines using "grep" command.The output displays the multiline string content before and after removing the empty lines.

3. Using “awk” Command

The “awk” is a powerful programming language for text processing and data extraction. The syntax is awk '!/^[[:space:]]*$/'.To remove empty lines from a text file using the awk command, you can check the following script:

#!/bin/bash

#define a multiline string
multiline_string="1st Line

2nd Line


3rd Line

4th Line"

echo "$multiline_string"
echo

multiline_string=$(echo "$multiline_string" | awk '!/^[[:space:]]*$/')

echo "After removing the empty lines:"
echo "$multiline_string"

EXPLANATION
The awk command uses !/^[[:space:]]*$/ pattern where ! reverses the pattern match. This pattern checks if a line does not consist solely of whitespace characters.

Bash remove the empty lines using the awk command.The script removes the empty lines and displays the multiline string before and after the removal.

Remove Leading Blank Lines Only

To remove leading empty lines using the “awk” command, follow the script below:

#!/bin/bash

# Define a multiline string with leading empty lines
multiline_string="



First Line
Second Line
Third Line"

# Remove leading empty lines using awk
new_string=$(awk 'BEGIN {p=0} !p && /^$/ {next} {print; p=1}' <<< "$multiline_string")

# Print the result
echo "$new_string"

EXPLANATION
This Bash script utilizes the awk command to remove leading empty lines from a multiline string. Within the awk command, BEGIN {p=0} initializes the variable p to 0 before processing any lines. !p && /^$/ {next} checks if p is not set (indicating leading empty lines) and if the current line is empty. If both conditions are true, it skips to the next line without printing it. {print; p=1} prints each line and sets p to 1 once a non-empty line is encountered, ensuring that only leading empty lines are removed.

Removes the leading empty lines from a multiline string using the "awk" command.The output shows the multiline string after removing the leading empty lines using the awk command.

Remove Trailing Blank Lines Only

To remove trailing empty lines using the “awk” command, follow the script below:

#!/bin/bash

# Define a multiline string
multiline_string="First Line
Second Line
Third Line



"

# Remove trailing empty lines using awk
new_string=$(awk 'NF || FNR != 1 {print}' <<< "$multiline_string")

# Print the result
echo "$new_string"

EXPLANATION
In this script, a multiline string is defined and stored in the variable multiline_string. The string contains several lines, including leading blank lines. In the awk command NF || FNR != 1 {print}, NF checks for non-empty lines, FNR tracks the line number, and FNR != 1 ensures it’s not the first line. {print} then outputs lines meeting these conditions, effectively removing trailing empty lines.

Removes the trailing empty lines from a multiline string in Bash.The output displays the multiline string after removing the trailing empty lines using the awk command.

4. Useing the “tr” Command

To delete empty lines using the tr command, you can use the tr command with the -S option to squeeze the repeated characters. The syntax is tr -s '\n'. Here’s how you can delete empty lines using tr:

#!/bin/bash

#define a multiline string
multiline_string="1st Line

2nd Line


3rd Line

4th Line"

#print the multiline string
echo "$multiline_string"
echo

#print the multiline string after re moving the empty lines
echo "After removing the spaces:"
echo "$multiline_string" | tr -s '\n'

EXPLANATION
This Bash script utilizes the tr command with the -s option to remove consecutive occurrences of newline characters (\n).

Bash remove the empty lines using the "tr" command.The output shows the multiline string with and without the empty lines.

5. Using “perl” Command

The perl command-line tool is a powerful utility for text processing and manipulation in Unix-like operating systems. The syntax is perl -ne 'print if /\S/' .

To remove the empty lines from a multiline string, check the following script:

#!/bin/bash

#define a multiline string
multiline_string="1st Line

2nd Line


3rd Line

4th Line"

#print the multiline stirng
echo "$multiline_string"
echo

#print the multiline string after removing the empty lines
echo "After removing the spaces:"
echo "$multiline_string" | perl -ne 'print if /\S/'

EXPLANATION
This Bash script defines a multiline string variable containing several lines of text, including empty ones. Then it pipes the multiline string into a Perl one-liner using the perl -ne command. Within the Perl one-liner, the -n option loops over each line of input, and the code print if /\S/ checks if the current line contains any non-whitespace character using the regex \S. If a line contains at least one non-whitespace character, it is printed to the standard output. Consequently, only lines with content are retained, effectively removing empty lines from the multiline string.

Remove the empty lines using the "perl".The output shows the multiline string before and after removing the empty lines using the “perl” command.

Pro Tips: Things to Consider While Removing Empty Lines

Here are a few tips to consider while removing empty lines from text or files:

  1. Consider backing up your files or original texts before removing the empty lines.
  2. You can create a new output file using the redirection operator (> or >>) to avoid the modification of the original file, text, or string.

Conclusion

To sum up, removing empty lines from text files or multiline strings is a common preprocessing task in Bash. By employing commands like ‘sed’, ‘grep’, ‘awk’, ‘tr’, and ‘perl’, you can effortlessly clean up your data and prepare it for further processing. I hope incorporating these techniques into your Bash scripts will undoubtedly enhance your workflow and help you achieve more efficiency in data manipulation tasks.

People Also Ask

How do I remove blank spaces in Linux?

To remove blank spaces from string or text, you can use parameter expansion. For example, check the following script to remove blank spaces from a string:

#!/bin/bash

# Define the input string with blank spaces
input_string="This is a string    with    blank    spaces."

# Remove blank spaces using parameter expansion
new_string="${input_string// /}"

echo "String after removing blank spaces:"
echo "$new_string"

##output
The string after removing the blank spaces:
Thisisasstringwithblankspaces

The script removes all the spaces from the string.

How to delete selected empty lines in a text file?

To delete selected empty lines of a text file, you can use the sed command.

Let’s consider a text file named file.txt whose contents are:

Linux 

Ubuntu 
Bash

Here, the line number is second. To remove that line, you can simply use the sed command with the 2d option as below:

sed '2d' file.txt

##output
Linux
Ubuntu
Bash

This is how you can delete any specific empty line from text files.

How to remove the last N lines of a File in Linux?

You can use the head command along with the -n option to specify the number of lines to keep. Here’s how you can remove the last N lines of a file in Bash.

Let’s take a file named example.txt with the content as below:

First
Second
Third
Fourth
Fifth
Sixth
Seventh

Check the script below to remove the last 3 lines of the example.txt file:

#!/bin/bash

# Define the filename and the number of lines to remove
filename="example.txt"
#Number of lines to remove
N=3


# Count the total number of lines in the file
total_lines=$(wc -l < "$filename")

# Calculate the number of lines to keep (total_lines - N)
lines_to_keep=$((total_lines - N))

# Use head to extract the first lines_to_keep lines and overwrite the 
head -n "$lines_to_keep" "$filename" > temp.txt && mv temp.txt "$filename"

##Output
First
Second
Third
Fourth

This script removes the last three lines from the text file.

How to remove the first line of a text file in Linux?

Bash offers the tail command which can be used with the -n option to skip the first line of a text file. Here’s how you can do it in Bash:

Let’s take a text file of name example.txt as below:

First
Second
Third
Fourth

To skip the first line of this text file, follow the script below:

#!/bin/bash

# Define the filename
filename="example.txt"

# Use tail to skip the first line and overwrite the original file
tail -n +2 "$filename" > temp.txt && mv temp.txt "$filename"
echo "$filename"

##output
Second
Third
Fourth

The script will override the original content of the example.txt file with the content starting from the second line.

Related Articles


<< Go Back to Bash Multiline StringBash String | Bash Scripting Tutorial

Rate this post
LINUX
FUNDAMENTALS
A Complete Guide for Beginners Enroll Course Now
icon linux
Auhona Islam

Auhona Islam is a dedicated professional with a background in Electronics and Communication Engineering (ECE) from Khulna University of Engineering & Technology. Graduating in 2023, Auhona is currently excelling in her role as a Linux content developer executive at SOFTEKO to provide a more straightforward route for Linux users. She aims to generate compelling materials for Linux users with her knowledge and skills. She holds her enthusiasm in the realm of Machine Learning (ML), Deep Learning (DL), and Artificial Intelligence (AI). Apart from these, she has a passion for playing instruments and singing. Read Full Bio

Leave a Comment