Binding stdout to stdin

Yes, that is really all that happens. It might be a little disappointing, now that you know all about input and output redirection. However, just because the concept is simple, that doesn't mean that pipes are not extremely powerful and very widely used.

Let's look at an example that shows how we can replace input/output redirection with a pipe:

reader@ubuntu:/tmp$ echo 'Fly into the distance' > file
reader@ubuntu:/tmp$ grep 'distance' < file
Fly into the distance
reader@ubuntu:/tmp$ echo 'Fly into the distance' | grep 'distance'
Fly into the distance

For the normal redirection, we first write some text to a file (using output redirection), which we then use as input for grep. Next, we do the exact same functional thing, but without the file as an intermediate step.

Basically, the pipe syntax is as follows:

command-with-output | command-using-input

You can use multiple pipes on a single line, and you can use any combination of pipes and input/output redirection, as long as it makes sense.

Often, when you get to the point of more than two pipes/redirections, you can increase readability with an extra line, perhaps using command substitution to write the intermediate result to a variable. But, technically, you can make it as complex as you want; just be vigilant in not making it too complicated.

As stated, pipes bind stdout to stdin. You might have an idea about the issue coming up: stderr! Look at this example of how the separation of output into stdout and stderr affects pipes:

reader@ubuntu:~/scripts/chapter_12$ cat /etc/shadow | grep 'denied'
cat: /etc/shadow: Permission denied
reader@ubuntu:~/scripts/chapter_12$ cat /etc/shadow | grep 'denied' > /tmp/empty-file
cat: /etc/shadow: Permission denied #Printed to stderr on terminal.
reader@ubuntu:~/scripts/chapter_12$ cat /etc/shadow | grep 'denied' 2> /tmp/error-file
cat: /etc/shadow: Permission denied #Printed to stderr on terminal.
reader@ubuntu:~/scripts/chapter_12$ cat /tmp/empty-file
reader@ubuntu:~/scripts/chapter_12$ cat /tmp/error-file

Now, initially this example might confuse you. Let's go through it step by step to figure it out.

First, cat /etc/shadow | grep 'denied'. We try to grep the stdout of cat /etc/shadow for the word denied. We do not actually find it, but we see it printed on our Terminal anyway. Why? Because even though stdout is piped to grep, stderr is sent straight to our Terminal (and not through grep).

If you're connecting via SSH to Ubuntu 18.04, you should see color highlighting by default when a grep is successful; in this example, you would not encounter this.

The next command, cat /etc/shadow | grep 'denied' > /tmp/empty-file, redirects the stdout of grep to a file. Since grep did not process the error message, the file remains empty.

Even if we try to redirect stderr at the end, as can be seen in the cat /etc/shadow | grep 'denied' 2> /tmp/error-file command, we still do not get any output in the file. This is because redirections are sequential: the output redirection only applies to the grep, not the cat.

Now, in the same way output redirections have a way to redirect both stdout and stderr, so does a pipe with the |& syntax. Look at the same example again, now using proper redirections:

reader@ubuntu:~/scripts/chapter_12$ cat /etc/shadow |& grep 'denied'
cat: /etc/shadow: Permission denied
reader@ubuntu:~/scripts/chapter_12$ cat /etc/shadow |& grep 'denied' > /tmp/error-file
reader@ubuntu:~/scripts/chapter_12$ cat /tmp/error-file
cat: /etc/shadow: Permission denied
reader@ubuntu:~/scripts/chapter_12$ cat /etc/shadow |& grep 'denied' 2> /tmp/error-file
cat: /etc/shadow: Permission denied
reader@ubuntu:~/scripts/chapter_12$ cat /tmp/error-file

For the first command, if you have color syntax enabled, you will see the word denied is bold and colored (in our case, red). This means that now that we use |&, grep did successfully process the output.

Next, when we redirect using the stdout of grep, we see that we successfully write the output to a file. If we try to redirect it with 2>, we see it printed in the Terminal again, but not in the file. This is because of the sequential nature of redirects: as soon as grep successfully processes the input (which came from stderr), grep outputs this to stdout.

grep actually does not know that the input was originally an stderr stream; as far as it is concerned, it is just stdin to process. And since for grep a successful process goes to stdout, that's where we find it in the end!

If we want to be safe and we do not need to split the functionality of stdout and stderr, the safest way would be to use the command like this: cat /etc/shadow |& grep 'denied' &> /tmp/file. Since both the pipe and output redirections process stdout and stderr, we'll always end up with all output where we want it.

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset
13.59.11.18