There are many reasons why you should learn about I/O redirection, shell filters, and shell expansion. These concepts are essential for anyone who wants to use the command line effectively. Without an understanding of how these features work, it will be difficult to fully utilize the potential of the command line. Additionally, learning about these topics will give you a better understanding of how Linux works under the hood. Finally, being knowledgeable about I/O redirection, shell filters, and shell expansion will make it easier for you to troubleshoot any issues that might arise when using the command line.
Redirection is a feature in Linux that allows you to change the standard input/output devices when executing a command. The basic workflow of any Linux command is that it takes input and produces output.
Each command performs all terminal-related activities with three files that the shell makes available to every command:
The > symbol is used for output redirection.
root@537302f1e700:~/fruits# ls -l > listings
root@537302f1e700:~/fruits# cat listings
total 0
-rw-r--r-- 1 root root 0 Sep 7 13:46 apple
-rw-r--r-- 1 root root 0 Sep 7 13:46 Apple
-rw-r--r-- 1 root root 0 Sep 7 13:49 banana
-rw-r--r-- 1 root root 0 Sep 7 13:48 Banana
-rw-r--r-- 1 root root 0 Sep 7 13:47 cherry
The output of ls when redirected will be stored in the listings file. This is how you can redirect a command.
When you use the ”>” operator to redirect to a file with the same name, the information in the file is overwritten.
root@537302f1e700:~/fruits# echo " welcome " > listings
root@537302f1e700:~/fruits# cat listings
welcome
In this example, echo "welcome" > listings has overwritten the existing file.
To prevent overwriting a file, you can append to it using the output append operator ».
root@537302f1e700:~/fruits# echo " welcome " > listings
root@537302f1e700:~/fruits# cat listings
welcome
root@537302f1e700:~/fruits# echo "welcome again" >> listings
root@537302f1e700:~/fruits# cat listings
welcome
welcome again
The ”<” symbol is used for stdin redirection.
For example:
$ wc -l users
2 users
$
$ wc -l < users
2
$
Note that there is a difference in the output produced by the two forms of the wc command. In the first case, the name of the file users is listed with the line count; in the second case, it is not.
In the first case, wc knows that it is reading its input from the file users. In the second case, it only knows that it is reading its input from standard input, so it does not display the filename.
Filters are programs that take in plain text as standard input, transform it into a meaningful format, and then return it as standard output. These include:
catThe cat command displays the text of the file line by line.
cat [path]
root@537302f1e700:~/fruits# cat list
apple
mango
banana
oranges
headThe head command displays the first n lines of a file. If the number of lines is not specified, it prints the first 10 lines by default.
head [lines-to-print] [path]
root@537302f1e700:~/fruits# head list
apple
mango
banana
oranges
Melon
mandarin
Avocado
coconut
Cherry
lime
root@537302f1e700:~/fruits# head -3 list
apple
mango
banana
root@537302f1e700:~/fruits#
tailThe tail command is the reverse of head, returning the lines from bottom to top.
root@537302f1e700:~/fruits# tail list
mango
banana
oranges
Melon
mandarin
Avocado
coconut
Cherry
lime
Longan
root@537302f1e700:~/fruits# tail -3 list
Cherry
lime
Longan
sortThe sort command is used to sort a file, arranging the records in a particular order.
sort [options] path
root@537302f1e700:~/fruits# sort list
1mango
20melon
3lime
apple
Avocado
banana
Cherry
coconut
Options with sort command:
uniqThe uniq command is used to remove duplicate lines. For example, in this case, banana was duplicated, and the uniq command removed the duplicate item.
root@537302f1e700:~/fruits# cat list
apple
mango
banana
banana
oranges
Melon
mandarin
Avocado
coconut
Cherry
lime
Longan
1mango
3lime
20melo
apple
root@537302f1e700:~/fruits# uniq list
apple
mango
banana
oranges
wcThe wc command gives the number of lines, words, and characters in the data. By default, it displays four columnar outputs.
root@537302f1e700:~/fruits# wc list
17 17 112 list
wc [options] [path]
wc stands for word count.
With wc, you can pass more than one file.
Note: When more than one file is placed in the argument, it adds one more line that counts the total.
root@537302f1e700:~/fruits# wc list melon orange
17 17 112 list
0 0 0 melon
0 0 0 orange
17 17 112 total
wc options:
wc.Some applications of the wc command:
ls list | wc -lroot@537302f1e700:~/fruits# ls list | wc -l
1
wc -w listroot@537302f1e700:~/fruits# wc -w list
17 list
grepThe grep command is used to search for particular information in a text file.
grep [options] pattern [path]
root@537302f1e700:~/fruits# grep banana list
banana
Or:
root@537302f1e700:~/fruits# cat list | grep banana
banana
grep options:
nlThe nl command is used to print line numbers of the text file.
nl [file-path]
root@537302f1e700:~/fruits# nl list
1 apple
2 mango
3 banana
4 oranges
5 Melon
6 mandarin
root@537302f1e700:~/fruits# cat -n list
1 apple
2 mango
3 banana
4 oranges
5 Melon
6 mandarin
7 Avocado
8 coconut
9 Cherry
10 lime
11 Longan
12 1mango
13 3lime
14 20melon
15 apple
16 mango
17 apple
nl options:
Some applications of nl command:
root@537302f1e700:~/fruits# head -5 list | nl -b a
1 apple
2 mango
3 banana
4 oranges
5 Melon
root@537302f1e700:~/fruits# head -n 3 list | nl -b a
1 apple
2 mango
3 banana
Shell expansion is the process of simplifying commands in order to get a return value from a shell prompt. The shell reads the command line and analyzes it to make it easier to understand.
The types of shell expansion include:
This is also known as “globbing.” Pathname expansion is performed by the shell when it encounters a wildcard pattern (e.g., *, ?, []). These wildcards are replaced with a list of matching files and directories.
For example, ls *.txt would list all .txt files in the current directory.
$ ls *.txt
file1.txt file2.txt file3.txt
When the shell encounters a tilde (~), it expands it to the current user’s home directory.
For example, cd ~ would change the directory to the user’s home directory.
$ echo ~
/home/username
Arithmetic expansion allows the evaluation of an arithmetic expression and its substitution into the command.
For example, echo $((2 + 2)) would output 4.
$ echo $((2 + 2))
4
Command substitution allows the output of a command to replace the command itself. This is done using backticks () or $(…)`.
For example, echo "Today is $(date)" would output the current date.
$ echo "Today is $(date)"
Today is Mon Aug 28 10:37:26 UTC 2024
Brace expansion is used to generate arbitrary strings. It is typically used for creating multiple similar files or directories at once.
For example, echo file{1..3}.txt would output file1.txt file2.txt file3.txt.
$ echo file{1..3}.txt
file1.txt file2.txt file3.txt
Variable expansion allows the value of a variable to replace the variable name.
For example, if NAME="John", then echo "Hello, $NAME" would output Hello, John.
$ NAME="John"
$ echo "Hello, $NAME"
Hello, John
Learning about I/O redirection, shell filters, and shell expansion is essential for effective command-line usage. These features allow for powerful text manipulation and automation, enabling users to work more efficiently with files, processes, and data in a Linux environment. By mastering these concepts, you will be better equipped to harness the full potential of the command line and streamline your workflow.