IN THIS CHAPTER
Based on the discussions in Chapter 3, “What Is the Shell?,” you should realize that whenever you type something like
who | wc -l
that you are actually programming in the shell! That's because the shell is interpreting the command line, recognizing the pipe symbol, connecting the output of the first command to the input of the second, and initiating execution of both commands.
In this chapter, you'll learn how to write your own commands and how to use shell variables.
A shell program can be typed directly at the terminal, as in
$ who | wc -l
or it can be first typed into a file and then the file can be executed by the shell. For example, suppose that you need to find out the number of logged-in users several times throughout the day. It's not unreasonable to type in the preceding pipeline each time you want the information, but for the sake of example, let's type this pipeline into a file. We'll call the file nu (for number of users), and its contents will be just the pipeline shown previously:
$ cat nu
who | wc –l
$
To execute the commands contained inside the file nu, all you now have to do is type nu as the command name to the shell: [1]
Oops! We forgot to mention one thing. Before you can execute a program this way, you must change the file's permissions and make it executable. This is done with the change mode command chmod. To add execute permission to the file nu, you simply type
chmod +x file(s)
The +x says make the file(s) that follow executable. The shell requires that a file be both readable and executable by you before you can execute it.
$ ls -l nu -rw-rw-r-- 1 steve steve 12 Jul 10 11:42 nu $ chmod +x nu Make it executable $ ls -l nu -rwxrwxr-x 1 steve steve 12 Jul 10 11:42 nu $
Now that you've made it executable, try it again:
$ nu
8
$
This time it worked.
You can put any commands at all inside a file, make the file executable, and then execute its contents simply by typing its name to the shell. It's that simple and that powerful.
The standard shell mechanisms such as I/O redirection and pipes can be used on your own programs as well:
$ nu > tally $ cat tally 8 $
Suppose that you're working on a proposal called sys.caps and that the following command sequence is needed every time you want to print a new copy of the proposal:
tbl sys.caps | nroff -mm –Tlp | lp
Once again, you can save yourself some typing by simply placing this command sequence into a file—let's call it run—making it executable, and then just typing the name run whenever you want to get a new copy of the proposal:
$ cat run tbl sys.caps | nroff -mm -Tlp | lp $ chmod +x run $ run request id is laser1-15 (standard input) $
(The request id message is issued by the lp command.) For the next example, suppose that you want to write a shell program called stats that prints the date and time, the number of users logged in, and your current working directory. You know that the three command sequences you need to use to get this information are date, who | wc -l, and pwd:
$ cat stats date who | wc –l pwd $ chmod +x stats $ stats Try it out Wed Jul 10 11:55:50 EDT 2002 13 /users/steve/documents/proposals $
You can add some echo commands to stats to make the output a bit more informative:
$ cat stats echo The current date and time is: date echo echo The number of users on the system is: who | wc –l echo echo Your current working directory is: pwd $ stats Execute it The current date and time is: Wed Jul 10 12:00:27 EDT 2002 The number of users on the system is: 13 Your current working directory is: /users/steve/documents/proposals $
Recall that echo without any arguments simply skips a line in the display. Shortly, you'll see how to have the message and the data displayed on the same line, like this:
The current date and time is: Wed Jul 10 12:00:27 EDT 2002
The shell programming language would not be complete without a comment statement. A comment is a way for you to insert remarks or comments inside the program that otherwise have no effect on its execution.
Whenever the shell encounters the special character # at the start of a word, it takes whatever characters follow the # to the end of the line as comments and simply ignores them. [2] If the # starts the line, the entire line is treated as a comment by the shell. Here are examples of valid comments:
# Here is an entire commentary line who | wc –l # count the number of users # # Test to see if the correct arguments were supplied #
Comments are useful for documenting commands or sequences of commands whose purposes may not be obvious or are sufficiently complex so that if you were to look at the program again in a week you might forget why they're there or what they do. Judicious use of comments can help make shell programs easier to debug and to maintain—both by you and by someone else who may have to support your programs.
Let's go back to the stats program and insert some comments:
$ cat stats
#
# stats -- prints: date, number of users logged on,
# and current working directory
#
echo The current date and time is:
date
echo
echo The number of users on the system is:
who | wc -l
echo
echo Your current working directory is:
pwd
$
The extra blank lines cost little in terms of program space yet add much in terms of program readability. They're simply ignored by the shell.
Like virtually all programming languages, the shell allows you to store values into variables. A shell variable begins with an alphabetic or underscore (_) character and is followed by zero or more alphanumeric or underscore characters.
To store a value inside a shell variable, you simply write the name of the variable, followed immediately by the equals sign =, followed immediately by the value you want to store in the variable:
variable=value
For example, to assign the value 1 to the shell variable count, you simply write
count=1
and to assign the value /users/steve/bin to the shell variable my_bin, you simply write
my_bin=/users/steve/bin
A few important points here. First, spaces are not permitted on either side of the equals sign. Keep that in mind, especially if you're in the good programming habit of inserting spaces around operators. In the shell language, you can't put those spaces in.
Second, unlike most other programming languages, the shell has no concept whatsoever of data types. Whenever you assign a value to a shell variable, no matter what it is, the shell simply interprets that value as a string of characters. So when you assigned 1 to the variable count previously, the shell simply stored the character 1 inside the variable count, making no observation whatsoever that an integer value was being stored in the variable.
If you're used to programming in a language such as C or Pascal, where all variables must be declared, you're in for another readjustment. Because the shell has no concept of data types, variables are not declared before they're used; they're simply assigned values when you want to use them.
As you'll see later in this chapter, the shell does support integer operations on shell variables that contain strings that are also valid numbers through special built-in operations.
Because the shell is an interpretive language, you can assign values to variables directly at your terminal:
$ count=1 Assign character 1 to count $ my_bin=/users/steve/bin Assign /users/steve/bin to my_bin $
So now that you know how to assign values to variables, what good is it? Glad you asked.
The echo command is used to display the value stored inside a shell variable. To do this, you simply write
echo $variable
The $ character is a special character to the shell. If a valid variable name follows the $, the shell takes this as an indication that the value stored inside that variable is to be substituted at that point. So, when you type
echo $count
the shell replaces $count with the value stored there; then it executes the echo command:
$ echo $count
1
$
Remember, the shell performs variable substitution before it executes the command (see Figure 5.1).
You can have the value of more than one variable substituted at a time:
$ echo $my_bin /users/steve/bin $ echo $my_bin $count /users/steve/bin 1 $
In the second example, the shell substitutes the value of my_bin and count and then executes the echo command (see Figure 5.2).
The values of variables can be used anywhere on the command line, as the next examples illustrate:
$ ls $my_bin mon nu testx $ pwd Where are we? /users/steve/documents/memos $ cd $my_bin Change to my bin directory $ pwd /users/steve/bin $ number=99 $ echo There are $number bottles of beer on the wall There are 99 bottles of beer on the wall $
Here are some more examples:
$ command=sort $ $command names Charlie Emanuel Fred Lucy Ralph Tony Tony $ command=wc $ option=-l $ file=names $ $command $option $file 7 names $
So you see, even the name of a command can be stored inside a variable. Because the shell performs its substitution before determining the name of the program to execute and its arguments, it scans the line
$command $option $file
and turns it into
wc -l names
Then it executes wc, passing the two arguments -l and names.
Variables can even be assigned to other variables, as shown in the next example:
$ value1=10 $ value2=value1 $ echo $value2 value1 Didn't do that right $ value2=$value1 $ echo $value2 10 That's better $
Remember that a dollar sign must always be placed before the variable name whenever you want to use the value stored in that variable.
What do you think happens when you try to display the value of a variable that has no value assigned to it? Try it and see:
$ echo $nosuch Never assigned it a value $
You don't get an error message. Did the echo command display anything at all? Let's see whether we can more precisely determine that:
$ echo :$nosuch: Surround its value with colons :: $
So you see no characters were substituted by the shell for the value of nosuch.
A variable that contains no value is said to contain the null value. It is the default case for variables that you never store values in. When the shell performs its variable substitution, any values that are null are completely removed from the command line, without a trace:
$ wc $nosuch -l $nosuch $nosuch names
7 names
$
The shell scans the command line substituting the null value for the variable nosuch. After the scan is completed, the line effectively looks like this:
wc -l names
which explains why it works.
Sometimes you may want to explicitly set a variable null in a program. This can be done by simply assigning no value to the variable, as in
dataflag=
Alternatively, you can list two adjacent pairs of quotes after the =. So
dataflag=""
and
dataflag=''
both have the same effect of assigning the null value to dataflag. Be advised that the assignment
dataflag=" "
is not equivalent to the three previous ones because it assigns a single space character to dataflag; that's different from assigning no characters to it.
Here's a puzzle for you: If you type
x=*
will the shell store the character * into the variable x, or will it store the names of all the files in your current directory into the variable x? Let's try it out and see:
$ ls What files do we have? addresses intro lotsaspaces names nu numbers phonebook stat $ x=* $ echo $x addresses intro lotsaspaces names nu numbers phonebook stat $
There's a lot to be learned from this small example. Was the list of files stored into the variable x when
x=*
was executed, or did the shell do the substitution when
echo $x
was executed?
The answer is that the shell does not perform filename substitution when assigning values to variables. Therefore,
x=*
assigns the single character * to x. This means that the shell did the filename substitution when executing the echo command. In fact, the precise sequence of steps that occurred when
echo $x
was executed is as follows:
The shell scanned the line, substituting
*as the value ofx.The shell rescanned the line, encountered the
*, and then substituted the names of all files in the current directory.The shell initiated execution of
echo, passing it the file list as arguments (see Figure 5.3).
This order of evaluation is important. Remember, first the shell does variable substitution, then does filename substitution, and then parses the line into arguments.
Suppose that you have the name of a file stored in the variable filename. If you wanted to rename that file so that the new name was the same as the old, except with an X added to the end, your first impulse would be to type
mv $filename $filenameX
When the shell scans this command line, it substitutes the value of the variable filename and also the value of the variable filenameX. The shell thinks filenameX is the full name of the variable because it's composed entirely of valid variable name characters. To avoid this problem, you can delimit the end of the variable name by enclosing the entire name (but not the leading dollar sign) in a pair of curly braces, as in
${filename}X
This removes the ambiguity, and the mv command then works as desired:
mv $filename ${filename}X
Remember that the braces are necessary only if the last character of the variable name is followed by an alphanumeric character or an underscore.
The POSIX standard shell provides a mechanism for performing integer arithmetic on shell variables called arithmetic expansion. Note that some older shells do not support this feature.
The format for arithmetic expansion is
$((expression))
where expression is an arithmetic expression using shell variables and operators. Valid shell variables are those that contain numeric values (leading and trailing whitespace is allowed). Valid operators are taken from the C programming language and are listed in Appendix A, “Shell Summary.”
The result of computing expression is substituted on the command line. For example,
echo $((i+1))
adds one to the value in the shell variable i and prints the result. Notice that the variable i doesn't have to be preceded by a dollar sign. That's because the shell knows that the only valid items that can appear in arithmetic expansions are operators, numbers, and variables. If the variable is not defined or contains a NULL string, its value is assumed to be zero. So if we have not assigned any value yet to the variable a, we can still use it in an integer expression:
$ echo $a Variable a not set $ $ echo $((a = a + 1)) Equivalent to a = 0 + 1 1 $ echo $a 1 Now a contains 1 $
Note that assignment is a valid operator, and the value of the assignment is substituted in the second echo command in the preceding example.
Parentheses may be used freely inside expressions to force grouping, as in
echo $((i = (i + 10) * j))
If you want to perform an assignment without echo or some other command, you can move the assignment before the arithmetic expansion.
So to multiply the variable i by 5 and assign the result back to i you can write
i=$(( i * 5 ))
Note that spaces are optional inside the double parentheses, but are not allowed when the assignment is outside them.
Finally, to test to see whether i is greater than or equal to 0 and less than or equal to 100, you can write
result=$(( i >= 0 && i <= 100 ))
which assigns result 1 if the expression is true and 0 if it's false:
$ i=$(( 100 * 200 / 10 )) $ j=$(( i < 1000 )) If i is < 1000, set j = 0; otherwise 1 $ echo $i $j 2000 0 i is 2000, so j was set to 0 $
That concludes our introduction to writing commands and using variables. The next chapter goes into detail on the quoting mechanisms in the shell.
[1] Note that the error produced here varies between different shells.
[2] Note that the # may be your default erase character. If so, to enter the character into an editor such as ed or vi, you'll have to “escape” it by preceding it with a . Alternatively, you can change your erase character to something else with the stty command.