One of the most common problems when writing shell scripts is following the often-cryptic syntax requirements. If you miss just one little thing, then the shell will fail to run your script.

# Has an error in a for loop.
#

for filename in *.doc
do
    echo "Copying $filename to $filename.bak"
    cp $filename $filename.bak
# done

echo "Completed backup operation on `date`."

$ sh debug_done
debug_done: line 11: syntax error: unexpected end of file

for loop started at line 4 without a done statement.

debug_done: line 11: syntax error: unexpected end of file

The next example shows a case where the shell is a bit more forthcoming about the detected problem.

# Shows an error.

echo "USER=$USER
echo "HOME=$HOME"
echo "OSNAME=$OSNAME"

$ sh debug_quotes
debug_quotes: line 6: unexpected EOF while looking for matching `"'
debug_quotes: line 8: syntax error: unexpected end of file

echo "USER=$USER

Another common source of scripting errors lies in simple typos—errors of some kind in the script. For example, forgetting something as simple as a space can create a lot of problems.

# Shows an error.
echo -n "Please enter the amount of purchase: "
read amount
echo

echo -n "Please enter the total sales tax: "
read rate
echo

if [$rate -lt 3 ]
then
    echo "Sales tax rate is too small."
fi

$ sh debug_sp
Please enter the amount of purchase: 100

Please enter the total sales tax: 7

debug_sp: line 11: [7: command not found

if [$rate -lt 3 ]

if [ $rate -lt 3 ]

The next example shows one of the errors most difficult to track down: a syntax error in a command, which the shell will not detect.

# Shows another error, harder to find.

# Using bc for math.
# Calculates sales tax.

echo -n "Please enter the amount of purchase: "
read amount
echo

echo -n "Please enter the total sales tax: "
read rate
echo

result=$( echo "
scale=2; tax=$amount*$rate/100.00;total=$amount+tax;print total" | bc )

echo "The total with sales tax is: $ $result."

$ sh debug_call
Please enter the amount of purchase: 100

Please enter the total sales tax: 7

(standard_in) 2: illegal character: 
The total with sales tax is: $ .

All of these examples are fairly simple. You can find the errors without a lot of work. That won't always be true when you are working with a larger shell script, especially a script that was written a while ago and is no longer fresh in anyone's memory. In addition, if someone else wrote the script, you have an even larger burden to decipher someone else's scripting style. To help solve script problems, you can try the following general-purpose techniques.

Start with the line number that the shell outputs for the error and work backward, toward the beginning of the script file. As shown previously, the line number reported by the shell often fails to locate the problem. That's because the shell cannot always track errors back to their sources. Therefore, you need to start with the reported error line and work backward, trying to find an error.

Typically, the shell detects an error, such as a missing double quote or ending statement for an if, for, while, or other construct. The shell's error message tells you where the shell detected the problem, often at the end of the file. You need to traverse backward toward the beginning of the file, looking for the missing item.

In many cases, the shell will help out by telling you what kind of item is missing, which can narrow the search considerably.

Look for syntax errors, typos, and other obvious mistakes. These types of errors are usually the easiest to find and fix.

For example, a typo in a variable name will likely not get flagged as an error by the shell but may well be problematic. In most cases, this will mean a variable is accessed (read) but never set to a value. The following script illustrates this problem:

echo -n "Please enter the amount of purchase: "
read amount
echo

echo -n "Please enter the total sales tax: "
read rate
echo

if [ $tax_rate -lt 3 ]
then
    echo "Sales tax rate is too small."
fi

The variable read in is rate, but the variable accessed is tax_rate. Both are valid variable names, but the tax_rate variable is never set.

No, this is not another front in the culture wars between conservative pundits and the rest of us. Instead, the goal is to focus your energies on any part of a script that looks strange. It doesn't matter what appears strange or whether the strangeness can be justified. Look for any part of the script that appears weird for any reason.

Of course, weird is another one of those technical terms sporting a nonprecise definition. All that can be said is that you'll know it when you see it. Moreover, as your experience with scripts grows, you'll be better able to separate the normal from the strange.

To help determine what is weird and what is not, use the following guidelines:

These guidelines may seem a bit strict, but they've proved useful in practice. Again, you're trying to identify areas of the script you should examine more closely, areas with a higher potential for holding the error or errors.

For example, not all Unix systems include a compiler for programs written in the C programming language. Sun's Solaris is rather well known for the lack of a general-purpose C compiler with its standard operating system. Scripts that assume all systems contain a C compiler are making an unjustified assumption. Such an assumption may be buried within the script, making it even harder to find.

Common assumptions include the following:

This technique worked for the ancient Romans; why not make it work for you? The divide-and-conquer technique is a last resort when you cannot narrow down the location of a problem. You'll use this most often when debugging long shell scripts written by others.

You start by choosing a location about halfway into the script file. You do not have to be exact. Stop the script at the halfway point with an exit statement. Run the script. Does the error occur? If so, you know the problem lies within the first half of the script. If not, then you know the problem lies within the last half of the script.

Next, you repeat the process in whichever half of the script appears to have the error. In this case, you divide the relevant half of the script in half again (into one-fourth of the entire script). Keep going until you find the line with the error.

This technique is simple, but it doesn't always work. Some scripts simply aren't appropriate for stopping at some arbitrary location. Nor is this technique appropriate if running part of the script will leave your system in an uncertain state—for example, starting a backup without completing the backup.

In this case, you can try a less aggressive approach. Instead of stopping the script at a dividing point, put in a read statement, as shown in the following example:

echo "Press enter to continue."
read ignored

This example requires the user to press the Enter key. The variable read in, ignored, is, you guessed it, ignored. The point of this snippet of a script is to pause the script. You can extend this technique by accessing the values of variables in the message passed to the echo command. This way, you can track the value of key variables through the script.

This technique is analogous to the divide-and-conquer method. See if you can break the script into small pieces. You want the pieces to be small enough that you can test each one independently of the others. By making each piece work, you can then make the entire script work, at least in theory.

In many cases, if you can break each piece of the script into a function, then you can test each function separately. (See Chapter 10 for more information about writing functions.) Otherwise, you need to extract a section of script commands into a separate script file.

In either case, you want to verify that the scripted commands work as expected. To test this, you need to provide the expected input and verify that the script section or function produces the required output. This can be tedious, so you may want to do this only for areas you've identified as likely error areas (using the other techniques in this section, of course).

Once you verify that all of the pieces work, or you fix them to work, you can start assembling the pieces. Again, take a step-by-step approach. You want to assemble just two pieces first and then add another, and another, and so on. The idea is to ensure that you always have a working script as you assemble the pieces back together. When finished, you should have a working version of the original script. Your script may now look a lot different from the original, but it should work.

This technique is a lot like a Dilbert cartoon with a tagline of "You be the computer now." You need to pretend to be the computer and step through the script. Start at the beginning, examining each statement. Essentially, you pretend you are the shell executing the script. For each statement, determine what the statement does and how that affects the script. You need to work through the script step by step.

Look over all the commands, especially each if statement, for loop, case statement, and so on. What you want to do is see how the script will really function. Often, this will show you where the error or errors are located. For example, you'll see a statement that calls the wrong command, or an if statement with a reversed condition, or something similar. This process is tedious, but usually, with time, you can track down problems.

While you step through the code, statement by statement, keep in mind the other techniques, especially the ones about looking for hidden assumptions and detecting weird things. If any statement or group of statements stands out, perform more investigation. For example, look up the online documentation for the commands. You can see if the script is calling the commands properly.

Following this advice literally may help you defeat biometric security, but actually you want more than the eyes. Ask another person to look at the script.

Start by describing the problem and then explain how you narrowed down the search to the area in which you think the error occurs. Then ask the person to look at the script. The goal of this technique is obvious: Often, another person can see what you have overlooked. This is especially true if you have been working at the problem for a while.

Don't feel embarrassed doing this, as top-level software developers use this technique all the time.

All of these techniques are manual techniques, however. You must perform all of the work yourself. While primitive, the shell does offer some help for debugging your scripts through the use of special command-line options.

The -n option, short for noexec (as in no execution), tells the shell to not run the commands. Instead, the shell just checks for syntax errors. This option will not convince the shell to perform any more checks. Instead, the shell just performs the normal syntax check. With the -n option, the shell does not execute your commands, so you have a safe way to test your scripts to see if they contain syntax errors.

The following example shows how to use the -n option.

$ sh -n debug_quotes
debug_quotes: line 6: unexpected EOF while looking for matching `"'
debug_quotes: line 8: syntax error: unexpected end of file

The -v option tells the shell to run in verbose mode. In practice, this means that the shell will echo each command prior to executing the command. This is very useful in that it can often help you find errors.

$ sh -v listusers

cut -d: -f1,5,7 /etc/passwd | grep -v sbin | grep sh | sort > users.txt

awk -F':' ' { printf( "%-12s %-40s
", $1, $2 )   } ' users.txt
ericfj       Eric Foster-Johnson
netdump      Network Crash Dump user
root         root

# Clean up the temporary file.
/bin/rm -rf users.txt

ericfj       Eric Foster-Johnson
netdump      Network Crash Dump user
root         root

You can combine the shell command-line options. Of these, the -n and -v options make a good combination because you can check the syntax of a script while seeing the script output.

The following example shows this combination.

$ sh -nv debug_quotes

# Shows an error.

echo "USER=$USER
echo "HOME=$HOME"
echo "OSNAME=$OSNAME"

debug_quotes: line 6: unexpected EOF while looking for matching `"'
debug_quotes: line 8: syntax error: unexpected end of file

The -x option, short for xtrace or execution trace, tells the shell to echo each command after performing the substitution steps. Thus, you'll see the value of variables and commands. Often, this alone will help diagnose a problem.

In most cases, the -x option provides the most useful information about a script, but it can lead to a lot of output. The following examples show this option in action.

$ sh -x listusers
+ cut -d: -f1,5,7 /etc/passwd
+ grep -v sbin
+ grep sh
+ sort
+ awk -F: ' { printf( "%-12s %-40s
", $1, $2 )   } ' users.txt
ericfj       Eric Foster-Johnson
netdump      Network Crash Dump user
root         root
+ /bin/rm -rf users.txt

The preceding example shows a relatively straightforward script. The following examples show slightly more complicated scripts.

if [ "$MYHOME" == "" ]
then
    # Check for Mac OS X home.
    if [ -d "/Users/$USER" ]
    then
        HOME="/Users/$USER"

    # Check for Linux home.
    elif [ -e "/home/$USER" ]
    then
        if [ -d "/home/$USER" ]
        then
           HOME="/home/$USER"
        fi
    else
        echo -n "Please enter your home directory: "
        read HOME
        echo
    fi
fi

$ sh -x nested_if
+ '[' '' == '' ']'
+ '[' -d /Users/ericfj ']'
+ '[' -e /home/ericfj ']'
+ '[' -d /home/ericfj ']'
+ HOME=/home/ericfj

# Using bc for math with a here document.
# Calculates sales tax.

echo -n "Please enter the amount of purchase: "
read amount
echo

echo -n "Please enter the total sales tax: "
read rate
echo

result=$(bc << EndOfCommands
scale=2   /* two decimal places */

tax = ( $amount * $rate ) / 100
total=$amount+tax

print total

EndOfCommands
)

echo "The total with sales tax is: $ $result on `date`."

$ sh -x trace_here
+ echo -n 'Please enter the amount of purchase: '
Please enter the amount of purchase: + read amount
100
+ echo

+ echo -n 'Please enter the total sales tax: '
Please enter the total sales tax: + read rate
7
+ echo

++ bc
+ result=107.00
++ date
+ echo 'The total with sales tax is: $ 107.00 on Thu Nov 25 07:51:36 CST 2004.'
The total with sales tax is: $ 107.00 on Thu Nov 25 07:51:36 CST 2004.

In the next example, you can see how the -x option tells the shell to output information about each iteration in a for loop. This is very useful if the loop itself contains a problem. The -x option enables you to better see how the script looks from the shell's point of view.

# Assumes $1, first command-line argument,
# names directory to list.

cd $1
for filename in *
do
    echo $filename
done

$ sh -x myls3 /usr/local
+ cd /usr/local
+ for filename in '*'
+ echo bin
bin
+ for filename in '*'
+ echo etc
etc

+ for filename in '*'
+ echo games
games
+ for filename in '*'
+ echo include
include
+ for filename in '*'
+ echo lib
lib
+ for filename in '*'
+ echo libexec
libexec
+ for filename in '*'
+ echo man
man
+ for filename in '*'
+ echo sbin
sbin
+ for filename in '*'
+ echo share
share
+ for filename in '*'
+ echo src
src

Because many script errors are caused by typos, you can format your script to make the syntax clearer. The following guidelines will not only make it easier to understand your scripts, but also help you see if they contain syntax errors:

This way, if the value of the directory ever changes, you have only one place in your script to change. Furthermore, your script is now less susceptible to typos. If you repeatedly type a long directory name, you may make a mistake. If you type the name just once, you are less likely to make a mistake.

The shell supports comments for a reason. Every script you write should have at least one line of comments explaining what the script is supposed to do.

In addition, you should comment all the command-line options and arguments, if the script supports any. For each option or argument, explain the valid forms and how the script will use the data.

These comments don't have to be long. Overly verbose comments aren't much help. However, don't use this as an excuse to avoid commenting altogether. The comments serve to help you, and others, figure out what the script does. Right now, your scripts are probably fresh in your memory, but six months from now you'll be glad you commented them.

Any part of your script that appears odd, could create an error, or contains some tricky commands merits extra comments. Your goal in these places should be to explain the rationale for the complex section of commands.

If the cryptic error messages from the shell impede your ability to debug scripts, then you shouldn't contribute to the problem. Instead, fight the Man and be a part of the solution. Create useful, helpful error messages in your scripts.

Error messages should clearly state the problem discovered, in terms the user is likely to understand, along with any corrective actions the user can take.

You may find that the error messages are longer than the rest of your script. That's okay. Error messages are really a part of your script's user interface. A user-friendly interface often requires a lot of commands.

Clever commands in your scripts show how clever you are, right? Not always, especially if one of your scripts doesn't work right. When faced with a script that's too clever for itself, you can focus on simplifying the script. Start with the most complicated areas, which are also likely to be the areas that aren't working. Then try to make simpler commands, if statements, for loops, case statements, and so on.

Often, you can extract script commands from one section into a function or two to further clarify the situation. The idea is to end up with a script that is easier to maintain over the long run. Experience has shown that simpler scripts are far easier to maintain than overly complicated ones.

Test your scripts. Test your scripts. Yep, test your scripts. If you don't test your scripts, then they will become examples for others of problematic debugging.

In many cases, especially for larger scripts, you may need to follow the techniques described in the section on breaking scripts into pieces. This concept works well for the scripts you write as well. If you can build your scripts from small, tested pieces, then the resulting whole is more likely to work (and more likely to be testable).

The only way you can determine whether your scripts work is to try them out.