What are Unix Pipe Commands with Examples?

In my previous article I have given the unix interview questions and answers. There is one of the important concept of Unix which is Unix Pipe Commands with its examples and answers. Unix pipe commands are one of the most powerful features of the Unix operating system. It allows you to combine simple command-line tools to create complex data processing workflows, without having to write any code. In this article, we will explore Unix pipelines and show you how they work, using examples.

What are Unix Pipe Commands with examples?

A Unix pipeline is a sequence of one or more commands connected by pipes, which allow the output of one command to be used as input to the next command. The basic syntax for a pipeline is as follows:

Example :

command1 | command2 | command3 …

In this example, the output of command1 is piped (|) to command2, and the output of command2 is piped to command3, and so on. The final output is the output of the last command in the pipeline.

Unix pipelines are often used for data processing, where you need to perform a series of operations on a large amount of data. By breaking down the data processing into smaller, simpler operations, you can create a pipeline that is more efficient and easier to manage.

Example 1: Counting Words in a File

Let’s start with a simple example of using a Unix pipeline to count the number of words in a file. We will use the following command:

cat file.txt | tr -s ‘ ‘ ‘\n’ | wc -w

Here is how this pipeline works:

1. cat file.txt: This command outputs the contents of the file.txt file to the terminal.
2. tr -s ' ' '\n': This command uses the tr command to replace all spaces (' ') with newlines ('\n'). This transforms the text into a list of words, with one word per line.
3. wc -w: This command uses the wc command to count the number of words (-w) in the input.

By combining these three commands into a pipeline, we can count the number of words in a file with just one command. Here’s an example of what the output might look like:

$ cat file.txt | tr -s ‘ ‘ ‘\n’ | wc -w
15

Example 2: Finding the Longest Word in a File

This is one of the important interview question of Unix. Let’s take the previous example a step further and use a Unix pipeline to find the longest word in a file. We will use the following command:

cat file.txt | tr -s ‘ ‘ ‘\n’ | awk ‘{ print length, $0 }’ | sort -rn | head -1 | cut -d” ” -f2-

Here is how this pipeline works:

1. cat file.txt: This command outputs the contents of the file.txt file to the terminal.
2. tr -s ' ' '\n': This command uses the tr command to replace all spaces (' ') with newlines ('\n'). This transforms the text into a list of words, with one word per line.
3. awk '{ print length, $0 }': This command uses the awk command to print the length of each line (word) followed by the line itself.
4. sort -rn: This command uses the sort command to sort the lines (words) in reverse numerical order (-rn) based on the length.
5. head -1: This command uses the head command to print the first line (word) of the sorted list, which will be the longest word.
6. cut -d" " -f2-: This command uses the cut command to remove the length value from the output, leaving only the longest word.

Example 3 : Find the top 10 largest files in a directory

To find the top 10 largest files in a directory, you can use the ‘ls’ command to list all the files, sort them by file size using the ‘sort’ command, and then display only the top 10 using the ‘head’ command. Here is the command:

ls -l | sort -k 5 -n -r | head -n 10

In this pipeline, the ‘ls -l’ command lists all the files in the current directory with their sizes, the ‘sort’ command sorts the files by size (-k 5) in descending order (-r), and the ‘head’ command displays the first 10 lines.

Example 4: Find all the files in a directory that contain a specific text

To find all the files in a directory that contain a specific text, you can use the ‘grep’ command to search for the text in each file and then display only the file names using the ‘cut’ command. Here is the command:

grep -r “search_text” /path/to/directory | cut -d : -f 1

In this pipeline, the ‘grep -r’ command searches recursively (-r) for the specified text in all files in the directory, and the ‘cut’ command extracts the file name (-f 1) from the output of the grep command.

I hope you got the exact idea about how Unix pipe commands work with examples. These pipe commands and most important in day to day working with Unix. If you like this article or if you have any issues with the same kindly comment in to comments section.