10.6. Redirection and Pipes
Normally, standard output (stdout) from a command goes to the screen, standard input (stdin) comes from the keyboard, and error messages (stderr) go to the screen. The shell allows you to use the special redirection metacharacters to redirect the input/output to or from a file. The redirection operators (<, >, >>, >&) are followed by a filename. This file is opened by the shell before the command on the left-hand side is executed.
Pipes, represented by a vertical bar (|) symbol, allow the output of one command to be sent to the input of another command. The command on the left-hand side of the pipe is called the writer because it writes to the pipe. The command on the right-hand side of the pipe is the reader because it reads from the pipe. See Table 10.15 for a list of redirection and pipe metacharacters.
10.6.1. Redirecting Input
Instead of the input coming from the terminal keyboard, it can be redirected from a file. The shell will open the file on the right-hand side of the < symbol and the program on the left will read from the file. If the file does not exist, the error "No such file or directory" will be reported by the C shell.
Format
command < file
Example 10.51.
mail bob < memo
|
Explanation
The file memo is opened by the shell, and the input is redirected to the mail program. Simply, the user bob is sent a file called memo by the mail program.
10.6.2. The Here Document
The here document is another way to redirect input to a command in the form of a quoted block of text. It is used in shell scripts for creating menus and processing input from other programs. Normally, programs that accept input from the keyboard are terminated with Control-D (^D). The here document provides an alternate way of sending input to a program and terminating the input without typing ^D. The << symbol is followed by a user-defined word, often called a terminator. Input will be directed to the command on the left-hand side of the << symbol until the user-defined terminator is reached. The final terminator is on a line by itself, and cannot be surrounded by any spaces. Variable and command substitution are performed within the here document. Normally, here documents are used in shell scripts to create menus and provide input to commands such as mail, bc, ex, ftp, etc. See Example 11.34.
Format
command << MARK
… input …
MARK
Example 10.52.
(Without the "Here" Document) (The Command Line) 1 > cat 2 Hello There. How are you? I'm tired of this. 3 ^d (The Output) 4 Hello There. How are you? I'm tired of this. |
Explanation
The cat program, without arguments, waits for keyboard input.
The user types input at the keyboard.
The user types ^D to terminate input to the cat program.
The cat program sends its output to the screen.
Example 10.53.
(With the "Here" Document) (The Command Line) 1 > cat << DONE 2 ? Hello There.? How are you? ? I'm tired of this. 3 ? DONE 4 Hello There. <----The output from the here document How are you? I'm tired of this. |
Explanation
The cat program will receive input from the first DONE to the terminating DONE. The words are user-defined terminators.
The question mark (?) is the secondary prompt, prompt2. It will appear until the here document is terminated with the user-defined terminator, DONE. These lines are input. When the word DONE is reached, no more input is accepted.
The final terminator marks the end of input. There cannot be any spaces on either side of this word.
The text between the first word DONE and the final word DONE is the output of the cat command (from here to here) and is sent to the screen. The final DONE must be against the left margin with no space or other text to the right of it.
Example 10.54.
(The Command Line) 1 > set name = steve 2 > mail $name << EOF 3 ? Hello there, $name 4 ? The hour is now `date +%H` 5 ? EOF 6 > |
Explanation
The variable name is expanded within the here document.
The question mark (?) is the secondary prompt, prompt2. It will appear until the here document is terminated with the user-defined terminator, EOF. The mail program will receive input until the terminator EOF is reached.
Command substitution is performed within the here document; that is, the command within the back quotes is executed and the output of the command is replaced within the string.
The terminator EOF is reached, and input to the mail program is stopped.
10.6.3. Redirecting Output
By default, the standard output of a command or commands normally goes to the terminal screen. To redirect standard output from the screen to a file, use the > symbol. The command is on the left-hand side of the > symbol, and a filename is on the right-hand side. The shell will open the file on the right-hand side of the > symbol. If the file does not exist, the shell will create it; if it does exist, the shell will open the file and truncate it. Often files are inadvertently removed when using redirection. (A special tcsh variable, called noclobber, can be set to prevent redirection from clobbering an existing file. See Table 10.16.)
format
command > file
Example 10.55.
cat file1 file2 > file3
|
Explanation
The contents of file1 and file2 are concatenated and the output is sent tofile3. Remember that the shell opens file3 before it attempts to execute the cat command. If file3 already exists and contains data, the data will be lost. If file3 does not exist, it will be created.
Appending Output to an Existing File
To append output to an existing file, use the >> symbol. If the file on the right-hand side of the >> symbol does not exist, it is created; if it does exist, the file is opened and output is appended to the end of the file.
Format
command >> file
Example 10.56.
date >> outfile
|
Explanation
The standard output of the date command is redirected and appended to outfile.
Redirecting Output and Error
The >& symbol is used to redirect both standard output and standard error to a file. Normally, a command is either successful and sends its output to stdout, or fails and sends its error messages to stderr. Some recursive programs, such as find and du, send both standard output and errors to the screen as they move through the directory tree. When you use the >& symbol, both standard output and standard error can be saved in a file and examined. The C shell does not provide a symbol for redirection of only standard error, but it is possible to get just the standard error by executing the command in a subshell. See Figure 10.3.
Figure 10.3. Redirecting stdout and stderr. See Example 10.57
Example 10.57.
1 > date Tue Aug 3 10:31:56 PDT 2000 2 > date >& outfile 3 > cat outfile Tue Aug 3 10:31:56 PDT 2000 |
Explanation
The output of the date command is sent to standard output, the screen.
The output and errors are sent to outfile.
Because there were no errors, the standard output is sent to outfile and the contents of the file are displayed.
Example 10.58.
1 > cp file1 file2 2 > cp file1 cp: missing destination file Try 'cp --help' for more information 3 > cp file1 >&errorfile 4 > cat errorfile cp: missing destination file Try 'cp --help' for more information |
Explanation
This time the destination file is missing and the cp command fails, sending an error to stderr, the terminal.
The >& symbol is used to send both stdout and stderr to errorfile. Because the only output from the command is the error message, that is what is saved in errorfile.
The contents of errorfile are displayed, showing that it contains the error message produced by the cp command.
Separating Output and Errors
Standard output and standard error can be separated by enclosing the command in parentheses. When a command is enclosed in parentheses, the C shell starts up a subshell, handles redirection from within the subshell, and then executes the command. By using the technique shown in Example 10.59, you can separate the standard output from the errors.
Example 10.59.
(The Command Line) 1 > find . -name '*.c' >& outputfile 2 > (find . -name'*.c' > goodstuff) >& badstuff |
Explanation
The find command is enclosed within parentheses. The shell will create a subshell to handle the command. Before creating the subshell, the words outside the parentheses will be processed; that is, the badstuff file will be opened for both standard output and error. When the subshell is started, it inherits the standard input, output, and errors from its parent. The subshell then has standard input coming from the keyboard, and both standard output and standard error going to the badstuff file. Now the subshell will handle the > operator. The stdout will be assigned the file goodstuff. The output is going to goodstuff, and the errors are going to badstuff. See Figure 10.4.
Figure 10.4. Separating stdout and stderr
The noclobber Variable
The special C shell built-in variable noclobber, when set, protects you from clobbering files with redirection. See Table 10.16.
| noclobber Is Not Set | File Exists | File Does Not Exist |
|---|---|---|
| command > file | File is overwritten. | File is created. |
| command >> file | File is appended to. | File is created. |
| noclobber Is Set | ||
| command > file | Error message. | File is created. |
| command >> file | File is appended to. | Error message. |
| Overwriting noclobber | ||
| command >! file | If the noclobber variable is set, override its effects for this command and either open or truncate file, redirecting output of command to file. | |
| command >>! file | Override noclobber variable; if file does not exist, it is created and output from command is appended to it. (See Example 10.60.) | |
Example 10.60.
1 > cat filex abc 123 2 > date > filex 3 > cat filex Tue Mar 18 11:51:04 PST 2000 4 > set noclobber 5 > date > filex filex: File exists. 6 > ls >! filex Override noclobber for this command only > cat filex abc ab1 dir filex plan.c 7 > ls > filex filex: File exists. 8 > date >> XXX XXX: No such file or directory. 9 > date >>! XXX Override noclobber for this command only 10 > unset noclobber Turn off noclobber permanently |
Explanation
The contents of filex are displayed on the screen.
The output of the date command is redirected to filex. The file is truncated and its original contents overwritten.
The contents of filex are displayed.
The noclobber variable is set.
Because filex already exists and noclobber is set, the shell reports that the file exists and will not allow it to be overwritten.
The effects of the >! symbol were temporary. It does not turn off noclobber. It simply overrides noclobber for the command where it is implemented.
Attempting to redirect and append the output of the date command to a nonexistent file causes an error message when noclobber is set.
The noclobber variable is overridden with the exclamation mark attached to the >> redirection symbol.
The noclobber variable is unset.