There are several Bash switches and options useful in tracking down problems in scripts.

The -n (no execution) switch checks a script for syntax errors without running it. Use this switch to check a script during development.

$ bash -n bad.sh
bad.sh: line 3: syntax error near unexpected token 'fi'
bad.sh: line 3: 'fi'

In this example, there is an error on or before line 3 of the script. The term token refers to a keyword or another piece of text near the source of the error.

The -o errexit option terminates the shell script if a command returns an error code. The exceptions are loops, so the if command cannot work properly if commands can't return a non-zero status. Use this option only on the simplest scripts without any other error handling; for example, it does not terminate the script if an error occurs in a subshell.

The -o nounset option terminates with an error if unset (or nonexistent) variables are referenced. This option reports misspelled variable names. nounset does not guarantee that all spelling mistakes will be identified (see Listing 8.1).

#!/bin/bash
#
# A simple script to list files

shopt -o -s nounset

declare -i TOTAL=0

let "TOTAL=TTOAL+1"         # not caught
printf "%s
" "$TOTAL"

if [ $TTOAL -eq 0 ] ; then  # caught
   printf "TOTAL is %s
" "$TOTAL"
fi

The -o xtrace option displays each command before it's executed. The command has all substitutions and expansions performed.

declare -i TOTAL=0
if [ $TOTAL -eq 0 ] ; then
   printf "%s
" "$TOTAL is zero"
fi

If this script fragment runs with the xtrace option, you should see something similar to the following results:

+ alias 'rm=rm -i'
+ alias 'cp=cp -i'
+ alias 'mv=mv -i'
+ '[' -f /etc/bashrc ']'
+ . /etc/bashrc
+++ id -gn
+++ id -un
+++ id -u
++ '[' root = root -a 0 -gt 99 ']'
++ umask 022
++ '[' '' ']'
+ declare -i TOTAL=0
+ '[' 0 -eq 0 ']'
+ printf '%s
' '0 is zero'
0 is zero

The first 11 lines are the commands executed in the profile scripts on the Linux distributions. The number of plus signs indicate how the scripts are nested. The last four lines are the script fragment after Bash has performed all substations and expansions. Notice the compound commands (like the if command) are left out (see Listing 8.2).

#!/bin/bash
#
# bad.bash: A simple script to list files

shopt -o -s nounset
shopt -o -s xtrace

declare -i RESULT
declare -i TOTAL=3

while [ $TOTAL -ge 0 ] ; do
  let "TOTAL—"
  let "RESULT=10/TOTAL"
  printf "%d
" "$RESULT"
done

xtrace shows the line-by-line progress of the script. In this case, the script contains a mistake resulting in an extra cycle through the while loop. Using xtrace, you can examine the variables and see that -ge should be replaced by -gt to prevent a cycle when TOTAL is zero.

$ bash bad.bash
+ declare -i RESULT
+ declare -i TOTAL=3
+ '[' 3 -ge 0 ']'
+ let TOTAL—
+ let RESULT=10/TOTAL
+ printf '%d
' 5
5
+ '[' 2 -ge 0 ']'
+ let TOTAL—
+ let RESULT=10/TOTAL
+ printf '%d
' 10
10
+ '[' 1 -ge 0 ']'
+ let TOTAL—
+ let RESULT=10/TOTAL
bad.sh: let: RESULT=10/TOTAL: division by 0 (error token is "L")
+ printf '%d
' 10
10
+ '[' 0 -ge 0 ']'
+ let TOTAL—
+ let RESULT=10/TOTAL
+ printf '%d
' -10
-10
+ '[' -1 -ge 0 ']'

You can change the trace plus sign prompt by assigning a new prompt to the PS4 variable. Setting the prompt to include the variable LINENO will display the current line in the script or shell function. In a script, LINENO displays the current line number of the script, starting with 1 for the first line. When used with shell functions at the shell prompt, LINENO counts from the first line of the function.

The built-in trap command (discussed in more detail in Chapter 10, “Job Control and Signals”) can be used to execute debugging commands after each line has been processed by Bash. Usually debug traps are combined with a trace to provide additional information not listed in the trace.

When debug trapping is combined with a trace, the debug trap itself is listed by the trace when it executes. This makes a printf rather redundant because the command is displayed with all variable substitutions completed prior to executing the printf. Instead, using the null command (:) displays variables without having to execute a shell command at all (see Listing 8.3).

#!/bin/bash
#
# debug_demo.sh : an example of a debug trap

trap ': CNT is now $CNT' DEBUG

declare -i CNT=0

while [ $CNT -lt 3 ] ; do
   CNT=CNT+1
done

When it runs with tracing, the value of CNT is displayed after every line.

$ bash -x debug_demo.sh
+ trap ': CNT is now $CNT' DEBUG
+ declare -i CNT=0
++ : CNT is now 0
+ '[' 0 -lt 3 ']'
++ : CNT is now 0
+ CNT=CNT+1
++ : CNT is now 1
+ '[' 1 -lt 3 ']'
++ : CNT is now 1
+ CNT=CNT+1
++ : CNT is now 2
+ '[' 2 -lt 3 ']'
++ : CNT is now 2
+ CNT=CNT+1
++ : CNT is now 3
+ '[' 3 -lt 3 ']'
++ : CNT is now 3

In a business environment, where money and careers are at stake, it's not enough to create a flawless program. There's always a chance some last-minute change will cause a program to produce the wrong result or even to crash. When that happens, the changes need to be undone or corrected as quickly as possible with no data loss.

A version control system is a program that maintains a master copy of the data files, scripts, and source programs. This master copy is kept in a directory called a repository. Every time a program is added or changed, it's resubmitted to the repository with a record of which changes were made, who made them, and when.

CVS (Concurrent Versions System) is the version control software supplied with most Linux distributions. Based on an older program called RCS (Revision Control System), CVS can share a script among multiple programmers and log any changes. It can work with individual files, whole directories, or large projects. They can be organized into groups of files called modules. CVS timestamps files, maintains version numbers, and identifies possible problems when two programmers update the same section of a program simultaneously.

CVS is also very popular for open source development. It can be configured to enable programmers all over the world to work on a project.

To use CVS, the project or team leader must create a directory to act as the version control repository as well as a subdirectory called CVSROOT. Then you define an environment variable called CVSROOT so CVS knows where to find the repository directory. For example, to make /home/repository the repository for your team, you set up the CVSROOT under Bash as follows

$ declare -rx CVSROOT=/home/repository

The repository holds copies of all the files, change logs, and other shared resources related to your project.

There are no special requirements for the software being added to the repository. However, when a program is added or updated, CVS reads through the file looking for special strings. If they are present, CVS replaces these strings with the latest information about this copy of the program. CVS documentation refers to these strings as keywords, although they are not keywords in the Bash sense.

The CVS keywords can be added anywhere in a script, but they should appear in a comment or in a quoted string if not in a comment. This prevents the keyword from being treated as an executable shell command.

# CVS: $Header$

When the script is added to the repository, CVS will fill in the header information.

# CVS: $Header: /home/repository/scripts/ftp.sh,v 1.1 2001/03/26
20:35:27 kburtch Exp $

The CVS keyword header should be placed in the header of a script.

#!/bin/bash
#
# flush.sh: Flush disks if nobody is on the computer
#
# Ken O. Burtch
# CVS: $Header$

CVS is controlled using the Linux cvs command. cvs is always followed by a CVS command word and any parameters for that command.

To add a new project directory to the CVS repository, use the import command. Import places the current directory's files in the repository under the specified name. Import also requires a short string to identify who is adding the project and a string to describe the state of the project. These strings are essentially comments and can be anything: Your login name and init-rel for initial release are good choices.

$ cvs import scripts kburtch init-rel

CVS starts your default text editor as indicated by the environment variables EDITOR or CVSEDITOR. CVS doesn't recognize the VISUAL variable. The file to be edited contains comment lines marked by a leading CVS:.

CVS: ___________________________________
CVS: Enter Log. Lines beginning with 'CVS: ' are removed automatically
CVS:
CVS: ___________________________________

When you are finished editing, CVS adds your program to the repository, recording your comments in the change log. The results are written to screen.

N scripts/ftp.sh
No conflicts created by this import

The N scripts/ftp.sh line indicates that CVS created a new project called scripts and added the Bash script ftp.sh to it. ftp.sh is now stored in the CVS repository, ready to be shared among the development team members. It is now safe to delete the project directory from your directory. In fact, it must be deleted before work can continue on the project.

Use the CVS command checkout (or co) to work on a project. This CVS command saves a copy of the project in the current directory. It also creates a CVS directory to save private data files used by CVS.

To use checkout, move to your home directory and type:

$ cvs checkout scripts
cvs checkout: Updating .
U scripts/ftp.sh

The subdirectory called scripts contains personal copies of project files from the repository. CVS maintains the original copy of ftp.sh. Another programmer can also checkout ftp.sh while you are working on your personal copy.

To add new files or directories to the project directory, use the add command. To add a file called process_orders.sh, use this:

$ cvs add process_orders.sh

As you work on your scripts, periodically check your work against the repository using the update command. If another programmer makes changes to the scripts, CVS will update your project directory to reflect the changes to the scripts. Any changes you have made, however, will not yet be added to the repository copies.

$ cvs update
cvs update: Updating .

Sometimes the changes involve the same part of the script and CVS can't combine the changes automatically. CVS calls this a conflict and notes this with a C during an update. CVS marks the place where the conflict occurred; you have to edit the script yourself to resolve the conflict.

If there are no other problems after an update, you can continue working on your source code.

To delete a script that is already in the repository, remove it using the rm command and perform a CVS update. CVS will delete the file.

While working on your source code, changes are not distributed to the rest of your team until you are ready. When the script is tested and you're ready to make it available, use commit (or ci, which stands for check in). Before committing changes, delete non-essential files (such as temporary files) to save space in the repository.

$ cvs commit

Like import, CVS commit commandstarts your editor and prompts you for a description of the changes you've made.

CVS commit also increments the version number of your changed scripts automatically. By convention, CVS begins numbering your project with 1.1. To start a new version 2.1, edit the version number in the $Header$ line (or any other CVS keywords in your script) and change the number to 2.0. CVS saves the script as version 2.1.

At any time, you can retrieve the log for a script or an entire project. The CVS log command displays all the related log entries, scripts, and version numbers:

$ cvs log project
cvs log: Logging project
RCS file: /home/repository/scripts/ftp.sh,v
Working file: scripts/ftp.sh
head: 1.1
branch: 1.1.1
locks: strict
access list:
symbolic names:
p1: 1.1.1.1
keyword substitution: kv
total revisions: 2; selected revisions: 2
description:
______________
revision 1.1
date: 1999/01/13 17:27:33; author: kburtch; state: Exp;
branches: 1.1.1;
Initial revision
______________
revision 1.1.1.1
date: 1999/01/13 17:27:33; author: kburtch; state: Exp; lines: +0 -0
Project started
=============================================================================

The status command gives you an overview of a project directory and a list of the scripts that have not been committed to the repository.

$ cvs status scripts
cvs status: Examining scripts
===================================================================
File: ftp.sh Status: Up-to-date
Working revision: 1.1.1.1 Wed Jan 13 17:27:33 1999
Repository revision: 1.1.1.1 /home/repository/scripts/ftp.sh,v
Sticky Tag: (none)
Sticky Date: (none)
Sticky Options: (none)

CVS has many other specialized capabilities not discussed here. Consult the CVS man page for further information.

The output of a command can be saved to a file with the tee command. The name symbolizes a pipe that splits into two at a T connection: A copy of the output is stored in a file without redirecting the original standard output. To capture both standard output and standard error, redirect standard error to standard output before piping the results to tee.

$ bash buggy_script.sh >& | tee results.txt

The tee —append (-a) switch adds the output to the end of an existing file. The —ignore-interrupts (-i) switch keeps tee running even if it's interrupted by a Linux signal.

This technique doesn't copy what is typed on standard input. To get a complete recording of a script's run, Linux has a script command. When a shell script is running under script, a file named typescript is created in the current directory. The typescript file is a text file that records a list of everything that appears in the shell session.

You can stop the recording process with the exit command.

$ script
Script started, file is typescript
$ bash buggy_script.sh
...
$ exit
exit
Script done, file is typescript

To test cron scripts without installing them under cron, use the watch command. Watch periodically re-runs a command and displays the results on the screen. Watch runs the command every two seconds, but you can specify a different number of seconds with the —interval= (or -n) switch. You can filter the results so that only differences are shown (—differences or -d) or so that all the differences so far are shown (—differences= cumulative).

There are two commands available for timing a program or script.

The Bash built-in command time tells you how long a program took to run. You can also use time to time a series of piped commands. Except for the real time used, the statistics returned by time refer to the system resources used by the script and not any of the commands run by the script.

The results are formatted according to the value of the TIMEFORMAT variable. The layout of TIMEFORMAT is similar to the date command formatting string in that it uses a set of % format codes.

The precision indicates the number decimal positions to show, with a default of 3. The character l (long) prints the value divided into minutes and seconds. If there is no TIMEFORMAT variable, Bash uses real %3lR user %3lU sys%3lS.

$ unset TIMEFORMAT
$ time ls > /dev/null

real    0m0.018s
user    0m0.010s
sys     0m0.010s
$ declare -x TIMEFORMAT="%P"
$ time ls > /dev/null
75.34
$ declare -x TIMEFORMAT="The real time is %lR"
$ time ls > /dev/null
The real time is 0m0.023s

Notice the times can vary slightly between the runs because other programs running on the computer affect them. To get the most accurate time, test a script several times and take the lowest value.

Linux also has a time command. This variation cannot time a pipeline, but it displays additional statistics. To use it, use the command command to override the Bash time.

$ command time myprog
3.09user 0.95system 0:05.84elapsed 69%CPU(0avgtext+0avgdata 0maxresident)k
0inputs+0outputs(4786major+4235minor)pagefaults 0swaps

Like Bash time, Linux time can format the results. The format can be stored in a variable called TIME (not TIMEFORMAT) or it can be explicitly indicated with the —format (-f) switch.

Statistics not relevant to your hardware are shown as zero.

$ command time grep ken /etc/aliases
Command exited with non-zero status 1
0.00user 0.00system 0:00.02elapsed 0%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (142major+19minor)pagefaults 0swaps
$ command time —format "%P" grep ken /etc/aliases
Command exited with non-zero status 1
0%
$ command time —format "Major faults = %F" grep ken /etc/aliases
Command exited with non-zero status 1
Major faults = 141

The —portability (-p) switch forces time to adhere to the POSIX standard, the same as Bash time –p, turning off many of the extended features.

$ command time —portability grep ken /etc/aliases
Command exited with non-zero status 1
real 0.00
user 0.00
sys 0.00

The results can be redirected to a file with —output (-o), or appended to a file with —append (-a). The —verbose (-v) option gives a detailed explanation of each statistic.

Linux man pages are special text files formatted for the groff program (GNU run off). groff is based on the older Unix programs troff (for printers) and nroff (for terminals). Troff was originally created in 1973 by Joseph F. Ossanna. By creating short man pages for important scripts, a single access point can be created for online information about your projects.

Place man pages for your own projects in section 9 of the manual. Originally, section 9 was used to document the Linux kernel, but it now follows the traditional Unix interpretation as being free for your personal use. Manual pages for section 9 are usually stored in /usr/share/man/man9. If you don't have access to this directory, make a man9 directory your home directory and store your personal manual pages there. By adding $HOME to the MANPATH environment variable in your Bash profile file, man will search the pages stored in your man9 directory.

$ mkdir ~/man9
$ declare -x MANPATH="$HOME:/usr/share/man "

man pages are text files containing groff markup codes (or macros) embedded in the text. These codes, much like HTML tags in a Web page, control the spacing, layout, and graphics used in the pages. You can also define your own groff codes. These codes always appear at the start of a line and begin with a period.

Here's an example of some groff markup codes:

."$Id$
.TH MAN 9 "25 July 1993" "Linux" "Nightlight Corporation Manual"
.SH NAME
ftp.sh - script to FTP orders to suppliers
.SH SYNOPSIS
.B ftp.sh
.I file

The groff codes begin on lines by themselves with a period (.), followed by a one- or two-letter code. For example, the .B code indicates that the text that follows is bold (similar to the HTML <b> tag).

The groff predefined macros pertaining to manual pages are documented in the section 7 manual page on man (man 7 man). Some of the more commonly used groff codes are as follows:

Although there are no firm requirements for a manual page, most man pages include one or more of the following sections: SYNOPSIS, DESCRIPTION, RETURN VALUES, EXIT STATUS, OPTIONS, USAGE, FILES, ENVIRONMENT, DIAGNOSTICS, SECURITY, CONFORMING TO, NOTES, BUGS, AUTHOR, and SEE ALSO. If you are using CVS, you can include a CVS keyword such as $Id$ in a VERSION section.

The easiest way to create a simple man page for your program is to find a similar man page and make a modified copy.

Listing 8.4 shows an example of a short, complete man page.

./"man page supply_ftp.sh.9
.TH "SUPPLY_FTP.SH" 9 "25 May 2001" "Nightlight" "Nightlight Corporation Manual"
.SH NAME
supply_ftp.sh - Bash script to FTP orders to suppliers
.SH SYNOPSIS
.B supply_ftp.sh
.I file
.I supplier
.SH DESCRIPTION
.B supply_ftp.sh
sends orders via FTP to suppliers.
.I file
is the name of the file to send.
.I supplier
is the name of the supplier.  The suppliers and their FTP account information
are stored in the text file
.I supplier.txt
.SH RETURN VALUES
The script returns 0 on a successful FTP and 1 if the FTP failed.
.SH AUTHOR
Created by Ken O. Burtch.
.SH FILES
/home/data/supplier.txt
.SH VERSION
$Id$

This page is displayed as

SUPPLY_FTP.SH(9)  Nightlight Corporation Manual  SUPPLY_FTP.SH(9)

NAME
       supply_ftp.sh - Bash script to FTP orders to suppliers

SYNOPSIS
       supply_ftp.sh file supplier

DESCRIPTION

       supply_ftp.sh sends orders via FTP to suppliers. file is the
       name of the file to send. supplier is the name of the supplier.
       The suppliers and their FTP account information are stored in
       the text file supplier.txt

RETURN VALUES
       The script returns 0 on a successful FTP and 1 if the FTP failed.

AUTHOR
       Created by Ken O. Burtch.

FILES
       /home/data/supplier.txt

VERSION
       $Id$

Nightlight                 25 May 2001                          1

$Id$ is updated when the page is committed using CVS.

The less command knows how to display a man page. Use this command to test a man page before installing it.

Some Linux distributions include a command called man2html that can convert a man page to a Web page. To convert a simple man page called my_man_page.9, type this

$ man2html < my_man_page.9 > my_man_page.html

Post the resulting page to your Web server.

The Linux diff command lists the changes between two or more files.

When used with the proper switches, diff creates a patch file containing a list of changes necessary to upgrade one set of files to another.

$ diff -u —recursive —new-file older_directory newer_directory > update.diff

For example, suppose you have a script to count the files in the current directory, as shown in Listing 8.5.

#!/bin/bash
#
# file_count: count the number of files in the current directory.
# There are no parameters for this script.

shopt -s -o nounset

declare -rx SCRIPT=${0##*/}            # SCRIPT is the name of this script
declare -rx ls="/bin/ls"               # ls command
declare -rx wc="/usr/bin/wc"           # wc command

# Sanity checks

if test -z "$BASH" ; then
  printf "Please run this script with the BASH shell
" >&2
  exit 192
fi
if test ! -x "$ls" ; then
  printf "$SCRIPT:$LINENO: the command $ls is not available — aborting
 " >&2
  exit 192
fi
if test ! -x "$wc" ; then
  printf "$SCRIPT: $LINENO: the command $wc is not available — aborting
 " >&2
exit 192
fi

ls -1 | wc -l

exit 0

You decide that the script is better with exit $? instead of exit 0. This code

$ diff -u —recursive —new-file older.sh newer.sh > file_count.diff

creates the patch file containing

@@ -26,5 +26,5 @@

 ls -1 | wc -l

-exit 0
+exit $?

The - indicates the exit 0 line will be removed. The + indicates that exit $? will be inserted. Then the old script will be upgraded to the new script.

The Linux patch command applies a patch file created by diff. Use the -p1 and -s switches.

$ cd older_directory
$ patch -p1 -s < update.diff

In the file_count script example, because the patch was created for one file and not for a directory, patch asks for the name of the older file to patch.

$ patch -p1 -s < file_count.diff
The text leading up to this was:
—————————————
|—- older.sh   Tue Feb 26 10:52:55 2002
|+++ newer.sh   Tue Feb 26 10:53:56 2002
—————————————
File to patch: older.sh

The file older.sh is now identical to newer.sh.

A shell archive (or a shar file) is a collection of text files or scripts encoded as a single shell script. The data in the scripts is represented here as files. Binary files are converted to text with the Linux uuencode command. Shell archives are self-extracting archives; when the shell script executes, the files in the archive are unpacked.

The Linux shar (shell archive) command is a utility to create new shell archives.

To save a file called orders.txt as a shell archive, use this

$ shar orders.txt > orders.shar
shar: Saving orders.txt (text)

To extract the file, run the shell archive with Bash.

$ bash orders.shar
x - creating lock directory
x - extracting orders.txt (text)

To create a shell archive of the files in the current directory, use this

$ shar * > myproject.shar

The shar command recursively archives any subdirectories when a directory is being archived.

There are a large number of switches for shar. These are primarily for handling special cases. A complete list appears in the reference section of this chapter.

There is also an unshar command. Not exactly the opposite of shar, this command extracts shar archives from a saved email message and then uses Bash to unpack the files.

Shell archives were a popular method of sharing files over newsgroups during the early days of the Internet. Shell archives are not particularly efficient, but they provide an example of an unusual use for shell scripts and support for them is available on all Linux distributions.

Although shell scripts might not run vertical bars down your screen and eject your disks as my C program did in my third-year O/S course, they can be equally as difficult to debug. Knowing that there are debugging features in Bash makes it much easier to find and repair scripts. With version control, patches, and transcripts, your work can be shared with other programmers and problems caused by updates to scripts can be quickly isolated. These tools come in handy in the next chapter on subshells.