pipe (|) operator in Linux command

pipe (|) operator in Linux command

Let’s Start understanding from the below example:

$ ifconfig | grep 192

inet 192.168.1.96  netmask 255.255.255.0  broadcast 192.168.1.255

Introduction to pipe (|)

One of the most powerful shell operators is the pipe (|). The pipe takes the output from one command and uses it as input for another. And, you’re not limited to a single piped command—you can stack them as many times as you like, or until you run out of output or file descriptors.

One of the main purposes of piping is filtering. You use piping to filter the contents of a large file—to find a particular string or word, for example. This purpose is why the most popular use for pipes involves the commands grep and sort. But, you’re not limited to those cases. You can pipe the output to any command that accepts stream input. Let’s look at a theoretical example as an illustration of how this process works:

$ cmd1 | cmd2

Both cmd1 and cmd2 are command-line utilities that output their results to the screen (stdout). When you pipe one command’s output to another, however, the information doesn’t produce output to the screen. The pipe redirects that output as input to cmd2.

Here are a few examples:

Let’s look at some real-world examples of how piping works.

Checking on NICs

Let’s say that you need to know if one of your network interface cards (NICs) has an IP address beginning with 192:

$ ifconfig | grep 192

inet 192.168.1.96  netmask 255.255.255.0  broadcast 192.168.1.255

You can also find out which live NICs you have on a system with a simple pipe to grep:

$ ifconfig | grep UP

enp0s3: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536

You could also grep for “RUNNING” or “RUN” to display the same information.

Examining permissions

Maybe you want to find out how many directories under /etc are writeable by root:

$ sudo ls -lR | grep drwx

The results are too long to list here, but as you can see from your displayed list, there are a lot of them. You still need to find out how many there are, and a visual count would take a long time. An easy option is to pipe the results of your ls command to the wc (word count) command:

$ sudo ls -lR | grep drwx | wc -l

285

Counting files

You don’t have to use grep all the time. For example, you can list the number of files in the /etc directory with this:

$ ls | wc -l

229

Again, your results might look different, but you know something is wrong if the command returns a small number of files.

Identifying processes

You can also perform complex tasks using pipes. To list the process IDs (PIDs) for all systemd-related processes:

$ ps -ef | grep systemd | awk '{ print $2 }'

1
471
500
608
631
7449
7494
17242

The awk command’s $2 output isolates the second (PID) column from the ps command. Note that the last entry, PID 17242, is actually the PID for the grep command displaying this information, as you can see from the full listing results here:

khess    17242  7505  0 09:40 pts/0    00:00:00 grep --color=auto systemd

To remove this entry from your results, use the pipe operator again:

$ ps -ef | grep systemd | awk '{ print $2 }' | grep -v grep

The -v switch tells the grep command to invert or ignore lines that contain the string that follows—in this case, any line containing the word “grep.”

Sorting results

Another popular use of the pipe operator is to sort your results by piping to the command, sort. Say that you want to sort the list of names in contacts.txt. First, let’s look at the contents as they are in the file before sorting:

$ cat contacts.txt

Bob Jones
Leslie Smith
Dana David
Susan Gee
Leonard Schmidt
Linda Gray
Terry Jones
Colin Doe
Jenny Case

Now, sort the list:

$ cat contacts.txt | sort

Bob Jones
Colin Doe
Dana David
Jenny Case
Leonard Schmidt
Leslie Smith
Linda Gray
Susan Gee
Terry Jones

You can reverse the sort order with the -r switch:

$ cat contacts.txt | sort -r

Terry Jones
Susan Gee
Linda Gray
Leslie Smith
Leonard Schmidt
Jenny Case
Dana David
Colin Doe
Bob Jones

Was the output for either of these what you expected? By default, the sort, the command performs a dictionary sort on the first word or column, which is why Bob Jones and Terry Jones are not listed one after the other.

You can sort by the second column with the -k switch, specifying the column you want to sort on, which in our list is column two:

$ cat contacts.txt | sort -k2

Jenny Case
Dana David
Colin Doe
Susan Gee
Linda Gray
Bob Jones
Terry Jones
Leonard Schmidt
Leslie Smith

If you have a long contact list and you think that this list contains duplicate entries, you can pipe your list to sort and then uniq to remove those duplicates:

$ cat contacts.txt | sort | uniq

This command only displays unique entries, but it doesn’t save the results. To sort the file, filter unique entries, and then save the new file, use this:

$ cat contacts.txt | sort -k2 | uniq > contact_list.txt

Remember that the pipe operator and the file redirect operators do different things.

Reference: https://www.redhat.com/