Objective:

•   3.3 Turning Commands into a Script

 A script is a program written in an interpreted language, typically associated with a shell or other program whose primary purpose is something other than as an interpreted language. In Linux, many scripts are shell scripts, which are associated with Bash or another shell. (If you’re familiar with batch files in DOS or Windows, scripts serve a similar purpose.) You can write shell scripts to help automate tedious repetitive tasks or to perform new and complex tasks. Many of Linux’s startup functions are performed by scripts, so mastering scripting will help you manage the startup process.

This chapter covers Bash shell scripts, beginning with the task of creating a new script file. It then describes several important scripting features that help you to perform progressively more complex scripting tasks.

Beginning a Shell Script

 Shell scripts are plain-text files, so you create them in text editors such as vi, nano, or pico, as described in Chapter 10, “Editing Files.” You shouldn’t use standard word processing programs, such as LibreOffice Writer, to create shell scripts as by default they embed binary code into the final document to indicate fonts, font sizes, and special characters. These binary codes confuse the Linux shell.

A shell script begins with a line that identifies the shell that’s used to run it, such as the following:

#!/bin/bash

 The first two characters are a special code that tells the Linux kernel that this is a script and to use the rest of the line as a pathname to the program that interprets the script. (This line is sometimes called the shebang, hashbang, hashpling, or pound bang line.) Shell scripting languages use a hash mark (#) as a comment character, so the script utility ignores this line, although the kernel doesn’t. On most systems, /bin/sh is a symbolic link that points to /bin/bash, but it can point to some other shell. Specifying the script as using /bin/sh guarantees that any Linux system will have a shell program to run the script, but if the script uses any features specific to a particular shell, you should specify that shell instead—for instance, use /bin/bash or /bin/tcsh instead of /bin/sh.

When you’re done writing the shell script, you should modify it so that it’s executable. You do this with the chmod command, which is described in more detail in Chapter 14, “Setting Ownership and Permissions.” For now, know that you use the a+x option to add execute permissions for all users. For instance, to make a file called my-script executable, you issue the following command:

$ chmod a+x my-script

You’ll then be able to execute the script by typing its name, possibly preceded by ./ to tell Linux to run the script in the current directory rather than searching the current path. If you fail to make the script executable, you can still run the script by running the shell program followed by the script name (as in bash my-script), but it’s generally better to make the script executable. If the script is one you run regularly, you may want to move it to a location on your path, such as /usr/local/bin. When you do that, you won’t have to type the complete path or move to the script’s directory to execute it; you can just type my-script.

Using Commands

One of the most basic features of shell scripts is the ability to run commands. You can use both commands that are built into the shell and external commands—that is, you can run other programs as commands. Most of the commands you type in a shell prompt are external commands; they’re programs located in /bin, /usr/bin, and other directories on your path. You can run such programs, as well as internal commands, by including their names in the script. You can also specify parameters to such programs in a script. For instance, suppose you want a script that launches two xterm windows and the KMail mail reader program. Listing 11.1 presents a shell script that accomplishes this goal.

Listing 11.1: A simple script that launches three programs

#!/bin/bash
/usr/bin/xterm &
/usr/bin/xterm &
/usr/bin/kmail &

Aside from the first line that identifies it as a script, the script looks just like the commands you might type to accomplish the task manually, except for one fact: the script lists the complete paths to each program. This is usually not strictly necessary, but listing the complete path ensures that the script will find the programs even if the PATH environment variable changes. On the other hand, if the program files move (say, because you upgrade the package from which they’re installed and the packager decides to move them), scripts that use complete paths will break. If a script produces a No such file or directory error for a command, typing which command, where command is the offending command, should help you locate it.

Each program-launch line in Listing 11.1 ends in an ampersand (&). This character tells the shell to go on to the next line without waiting for the first to finish. If you omit the ampersands in Listing 11.1, the effect will be that the first xterm will open but the second won’t open until the first is closed. Likewise, KMail won’t start until the second xterm terminates.

Although launching several programs from one script can save time in startup scripts and some other situations, scripts are also frequently used to run a series of programs that manipulate data in some way. Such scripts typically do not include the ampersands at the ends of the commands because one command must run after another or may even rely on output from the first. A comprehensive list of such commands is impossible because you can run any program you can install in Linux as a command in a script—even another script. A few commands that are commonly used in scripts include the following:

Normal File Manipulation Commands  The file manipulation commands, such as ls, mv, cp, and rm, are often used in scripts. You can use these commands to help automate repetitive file maintenance tasks.

grep  The grep command locates files that contain the string you specify, or it displays the lines that contain those strings in a single file. grep is described in detail in Chapter 8, “Searching, Extracting, and Archiving Data.”

find  The find command searches for patterns based on filenames, ownership, and similar characteristics. Chapter 8 covers this command.

cut  The cut command extracts text from fields in a file. It’s frequently used to extract variable information from a file whose contents are highly patterned. To use it, you pass it one or more options that specify what information you want, followed by one or more filenames. For instance, users’ home directories appear in the sixth colon-delimited field 
of the /etc/passwd file. You can therefore type cut -f 6 -d ":" /etc/passwd to extract this information. The same command in a script will extract this information, which you’ll probably save to a variable or pass to a subsequent command.

sed  The sed program provides many of the capabilities of a conventional text editor (such as search-and-replace operations) but via commands that can be typed at a command prompt or entered in a script.

 echo  The echo command is the tool to use when a script must provide a message to the user. You can pass various options to echo or just a string to be shown to the user. For instance, echo "Press the Enter key" causes a script to display the specified string. You can also use echo to display the value of variables (described later, in “Using Variables”).

mail  The mail command can be used to send email from within a script. Pass it the 
-s subject parameter to specify a subject line, and give it an email address as the last argument. If it’s used at the command line, you then type a message and terminate it with a Ctrl+D keystroke. If it’s used from a script, you may omit the subject entirely or pass it an external file as the message using input redirection. You may want to use this command to send mail to the superuser about the actions of a startup script or a script that runs on an automated basis.

Many of these commands are extremely complex, and completely describing them is beyond the scope of this chapter. You can consult these commands’ man pages for more information. A few of them are described elsewhere in this book, as noted in their descriptions.

Even if you have a full grasp of how to use some key external commands, simply executing commands as you would when typing them at a command prompt is of limited utility. Many administrative tasks require you to modify what you type at a command, or even what commands you enter, depending on information from other commands. For this reason, scripting languages include additional features to help you make your scripts useful.

Using Arguments

 Variables can help you expand the utility of scripts. A variable is a placeholder in a script for a value that will be determined when the script runs. Variables’ values can be passed as parameters to a script, generated internally to a script, or extracted from a script’s environment. (An environment is a set of variables that any program can access. The environment includes things like the current directory and the search path for running programs.)

Variables that are passed to the script are frequently called parameters or arguments. They’re represented in the script by a dollar sign ($) followed by a number from 0 up—$0 stands for the name of the script, $1 is the first parameter to the script, $2 is the second parameter, and so on. To understand how this might be useful, consider the task of adding a user. As described in Chapter 13, “Creating Users and Groups,” creating an account for a new user typically involves running at least two commands—useradd and passwd. You may also need to run additional site-specific commands, such as commands that create unusual user-owned directories aside from the user’s home directory.

As an example of how a script with an argument variable can help in such situations, consider Listing 11.2. When you run the script, you must provide the user account as a parameter on the command line. The script retrieves that value using the $1 variable and creates an account based on the value with the useradd command. It then changes the account’s password using the passwd command (the script prompts you to enter the password when you run the script). It creates a directory in the /shared directory tree corresponding to the account, and it sets a symbolic link to that directory from the new user’s home directory. It also adjusts ownership and permissions in a way that may be useful, depending on your system’s ownership and permissions policies.

Listing 11.2: A script that reduces account-creation tedium

#!/bin/bash
useradd -m $1
passwd $1
mkdir -p /shared/$1
chown $1.users /shared/$1
chmod 775 /shared/$1
ln -s /shared/$1 /home/$1/shared
chown $1.users /home/$1/shared

If you use Listing 11.2, you need to type only three things: the script name with the desired username and the password (twice). For instance, if the script is called mkuser, you can use it like this:

#  mkuser rblum

Changing password for user rblum
New password:
Retype new password:
passwd: all authentication tokens updated successfully

Most of the script’s programs operate silently unless they encounter problems, so the interaction (including typing the passwords, which don’t echo to the screen) is a result of just the passwd command. In effect, Listing 11.2’s script replaces seven lines of commands with one. Every one of those lines uses the username, so by running this script, you also reduce the chance of a typo causing problems.

Using Variables

 Another type of variable that can be set from the output of a command is also identified by leading dollar signs, but typically is given a name that at least begins with a letter, such as $Addr or $Name. (When values are assigned to variables, the dollar sign is omitted, as illustrated shortly.) You can then use these variables with normal commands as if they were command parameters, but the value of the variable is passed to the command.

Consider Listing 11.3, which checks to see if the computer’s router is up with the help of the ping utility. This script uses two variables. The first is $ip, which is extracted from the output of route using the grep, tr, and cut commands. When you assign a value to a variable from the output of a command, that command should be enclosed in backtick characters (`), which appear on the same key as the tilde (~) on most keyboards. These are not ordinary single quotes, which appear on the same key as the regular quote character (") on most keyboards. The second variable, $ping, simply points to the ping program. It can just as easily be omitted, with subsequent uses of $ping replaced by the full path to the program or simply by ping (relying on the $PATH environment variable to find the program). Variables like this are sometimes used to make it easier to modify the script in the future. For instance, if you move the ping program, you need to modify only one line of the script. Variables can also be used with conditionals to ensure that the script works on more 
systems—for instance, if ping were called something else on some systems.

Listing 11.3: Script demonstrating assignment and use of variables

#!/bin/bash
ip=`route -n | grep UG | tr -s " " | cut -f 2 -d " "`
ping="/bin/ping"
echo "Checking to see if $ip is up..."
$ping -c 5 $ip

In practice, you use Listing 11.3 by typing the script’s name. The result should be the message Checking to see if 192.168.1.1 is up (with 192.168.1.1 replaced by the computer’s default gateway system) and the output from the ping command, which should attempt to send five packets to the router. If the router is up and is configured to respond to pings, you’ll see five return packets and summary information, similar to the following:

$ routercheck

Checking to see if 192.168.1.1 is up...
PING 192.168.1.1 (192.168.1.1) 56(84) bytes of data.
64 bytes from 192.168.1.1: icmp_seq=1 ttl=63 time=23.0 ms
64 bytes from 192.168.1.1: icmp_seq=2 ttl=63 time=0.176 ms
64 bytes from 192.168.1.1: icmp_seq=3 ttl=63 time=0.214 ms
64 bytes from 192.168.1.1: icmp_seq=4 ttl=63 time=0.204 ms
64 bytes from 192.168.1.1: icmp_seq=5 ttl=63 time=0.191 ms
 
--- 192.168.1.1 ping statistics ---
5 packets transmitted, 5 received, 0% packet loss, time 4001ms
rtt min/avg/max/mdev = 0.176/4.758/23.005/9.123 ms

If the router is down, you’ll see error messages to the effect that the host was unreachable.

Listing 11.3 is of limited practical use and contains bugs. For instance, the script identifies the computer’s gateway merely by the presence of the string UG in the router’s output line from route. If a computer has two routers defined, this won’t work correctly, and the result is likely to be a script that misbehaves. The purpose of Listing 11.3 is to illustrate how variables can be assigned and used, not to be a flawless working script.

Scripts like Listing 11.3, which obtain information from running one or more commands, are useful in configuring features that rely on system-specific information or information that varies with time. You can use a similar approach to obtain the current hostname (using the hostname command), the current time (using date), the total time the computer’s been running (using uptime), free disk space (using df), and so on. When combined with conditional expressions (described shortly), variables become even more powerful because then your script can perform one action when one condition is met and another in some other case. For instance, a script that installs software can check free disk space and abort the installation if insufficient disk space is available.

 In addition to assigning variables with the assignment operator (=), you can read variables from standard input using read, as in read response to read input for subsequent access as $response. This method of variable assignment is useful for scripts that must interact with users. For instance, instead of reading the username from the command line, Listing 11.2 may be modified to prompt the user for the username. Listing 11.4 shows the result. To use this script, you type its name without typing a username on the command line. The script will then prompt for a username, and after you enter one, the script will attempt to create an account with that name.

Listing 11.4: Modified version of Listing 11.2 that employs user interaction

#!/bin/bash
echo -n "Enter a username: "
read name
useradd -m $name
passwd $name
mkdir -p /shared/$name
chown $name.users /shared/$name
chmod 775 /shared/$name
ln -s /shared/$name /home/$name/shared
chown $name.users /home/$name/shared

One special type of variable is an environment variable, which is assigned and accessed just like a shell script variable. The difference is that the script or command that sets an environment variable uses Bash’s export command to make the value of the variable accessible to programs launched from the shell or shell script that made the assignment. In other words, you can set an environment variable in one script and use it in another script that the first script launches. Environment variables are most often set in shell startup scripts, but the scripts you use can access them. For instance, if your script calls X programs, it might check for the presence of a valid $DISPLAY environment variable and abort if it finds that this variable isn’t set. By convention, environment variable names are all uppercase, whereas non-environment shell script variables are all lowercase or mixed case.

One special variable deserves mention: $?. This variable holds the exit status (or return value) of the most recently executed command. The exit status is an integer value that you can check to determine whether or not the command completed correctly. Most programs return a value of 0 when they terminate normally and return another value to specify errors. You can display this value with echo or use it in a conditional expression (described next) to have your script perform special error handling.

Using Conditional Expressions

 Scripting languages support several types of conditional expressions. These enable a script to perform one of several actions contingent on some condition—typically the value of a variable. One common command that uses conditional expressions is if, which allows the system to take one of two actions depending on whether some condition is true. The conditional expression for the if keyword appears in brackets after the if keyword and can take many forms. Conditional expressions use options and operators to define just what condition to check. For instance, the condition:

[ -f 
file ]

uses the -f option and is true if file exists and is a regular file; whereas the condition:

[ -s 
file ]

uses the -s option and is true if file exists and has a size greater than 0. You can also use some string operators in conditions. For example:

[string1 == string2 ]

uses the == operator and is true if the two strings have the same values.

 To better understand the use of conditionals, consider the following code fragment:

if [ -s /tmp/tempstuff ]
   then
      echo "/tmp/tempstuff found; aborting!"
      exit
fi

This fragment causes the script to exit if the file /tmp/tempstuff is present. The then keyword marks the beginning of a series of lines that execute only if the conditional is true, and fi (if backward) marks the end of the if block. Such code may be useful if the script creates and then later deletes this file, because its presence indicates that a previous run of the script didn’t succeed or is still underway.

 An alternative form for a conditional expression uses the test keyword rather than square brackets around the conditional:

if test -s /tmp/tempstuff

You can also test a command’s return value by using the command as the condition:

if [  command ]
   then
      additional-commands

fi

In this example, the additional-commands will be run only if command completes successfully. If command returns an error code, additional-commands won’t be run.

Conditional expressions may be expanded by use of the else clause:

if [  conditional-expression ]
   then
      commands

   else
      other-commands

fi

Code of this form causes either commands or other-commands to execute, depending on the evaluation of conditional-expression. This is useful if something should happen in a part of the program, but precisely what should happen depends on some condition. For instance, you may want to launch one of two different file archiving programs depending on a user’s input.

 What do you do if more than two outcomes are possible—for instance, if a user may provide any one of four possible inputs? You can nest several if/then/else clauses, but this gets awkward quickly. A cleaner approach is to use case:

case  word in
   pattern1) command(s) ;;  
    pattern2) command(s) ;;
   ...
esac

 For a case statement, a word is likely to be a variable, and each pattern is a possible value of that variable. The patterns can be expanded much like filenames, using the same wildcards and expansion rules (* to stand for any string, for instance). You can match an arbitrary number of patterns in this way. Each set of commands must end with a double semicolon (;;), and the case statement as a whole ends in the string esac (case backward).

Upon execution, bash executes the commands associated with the first pattern to match the word. Execution then jumps to the line following the esac statement; any intervening commands don’t execute. If no patterns match the word, no code within the case statement executes. If you want to have a default condition, use * as the final pattern; this pattern matches any word, so its commands will execute if no other pattern matches.

Using Loops

Conditional expressions are sometimes used in loops. Loops are structures that tell the script to perform the same task repeatedly until some condition is met (or until some condition is no longer met). For instance, Listing 11.5 shows a loop that plays all the WAV audio files in a directory.

Listing 11.5: A script that executes a command on every matching file in a directory

#!/bin/bash
for d in `ls *.wav` ; do
   aplay $d
done

The for loop as used here executes once for every item in the list generated by ls *.wav. Each of those items (filenames) is assigned in turn to the $d variable and so is passed to the aplay command.

The seq command can be useful in creating for loops (and in other ways, too). This command generates a list of numbers starting from its first argument and continuing to its last one. For instance, typing seq 1 10 generates 10 lines, each with a number between 1 and 10. You can use the seq command in a for loop to iterate through a series of numbers:

for x in `seq 1 10` ; do
    echo $x
done

The loop executes 10 times, with the value of x incrementing with each iteration. If you pass just one parameter to seq, it interprets that number as an ending point, with the starting point being 1. If you pass three values to seq, it interprets them as a starting value, an increment amount, and an ending value.

 Another type of loop is the while loop, which executes for as long as its condition is true. The basic form of this loop type is:

while [  condition ]
do
  commands

done

The until loop is similar in form, but it continues execution for as long as its condition is false—that is, until the condition becomes true.

Using Functions

A function is a part of a script that performs a specific subtask and that can be called by name from other parts of the script. Functions are defined by placing parentheses after the function name and enclosing the lines that make up the function within curly braces:

myfn() {
   commands

}

The keyword function may optionally precede the function name. In either event, the function is called by name as if it were an ordinary internal or external command.

Functions are very useful in helping to create modular scripts. For instance, if your script needs to perform half a dozen distinct computations, you can place each computation in a function and then call them all in sequence. Listing 11.6 demonstrates the use of functions in a simple program that copies a file but aborts with an error message if the target file already exists. This script accepts a target and a destination filename and must pass those filenames to the functions.

Listing 11.6: A script demonstrating the use of functions

#/bin/bash
 
doit() {
   cp $1 $2
}
 
function check() {
  if [ -s $2 ]
     then
        echo "Target file exists! Exiting!"
        exit
  fi
}
 
check $1 $2
doit $1 $2

If you enter Listing 11.6 and name it safercp, you can use it like this, assuming the file original.txt exists and dest.txt doesn’t:

$ ./safercp original.txt dest.txt

$ ./safercp original.txt dest.txt

Target file exists! Exiting!

The first run of the script succeeded because dest.txt didn’t exist. On the second run, though, the destination file did exist, so the script terminated with the error message.

Note that the functions aren’t run directly and in the order in which they appear in the script. They’re run only when called in the main body of the script—which in Listing 11.6 consists of just two lines, each corresponding to one function call, at the very end of the script.

Setting the Script’s Exit Value

 Ordinarily, a script’s exit status is the same as the last command the script called—that is, the script returns $?. You can control the exit value, however, or exit from the script at any point, by using the exit command. Used without any options, exit causes immediate termination of the script, with the usual exit value of $?. This can be useful in error handling or in aborting an ongoing operation for any reason—if the script detects an error or if the user selects an option to terminate, you can call exit to quit.

If you pass a numeric value between 0 and 255 to exit, the script terminates and returns the specified value as the script’s own exit value. You can use this feature to signal errors to other scripts that might call your own script. You may have to include extra code to keep track of the causes of abnormal termination, though. For instance, you can set aside a variable (say, $termcause) to hold the cause of the script’s termination. Set it to 0 at the start of the script and then, if the script detects a problem that will cause termination, reset $termcause to some non-0 value. (You can use any numeric codes you like; there’s no set meaning for such codes.) On exit, be sure to pass $termcause to exit:

exit $termcause

Summary

Serious Linux users and administrators must have at least a basic understanding of shell scripts. Many configuration and startup files are in fact shell scripts, and being able to read them, and perhaps modify them, will help you administer your system. Being able to create new shell scripts is also important, because doing so will help you simplify tedious tasks and create site-specific tools by gluing together multiple programs to accomplish your goals. Linux shell scripts allow you to combine standard shell commands with a few specialized programming commands, such as conditional expressions, loops, and functions, to implement logic features often found in larger-scale programs.

Exam Essentials

Describe the format of a basic shell script file.  Shell script files must be plain-text files; they cannot be generated using a word processing application. The first line in the shell script should be the shebang combination (#!/bin/sh). This indicates the shell that should be used to run the script. The remainder of the shell script is a list of commands or shell statements arranged in the order in which they are to be run by the shell. Though not required, it’s common to use the exit statement at the end of a shell script to control the exit status generated by the shell, especially if the shell exists with any type of error condition.

Explain how using variables helps when writing a shell script.  Shell scripts use variables to store data or other information for use later in the shell script. You assign data to a variable using the equal sign (=). The data can be any type of text or numerical data, or if you redirect the output of a command to store in a variable by placing backtick characters around the command. To reference the value of a variable inside the shell script, precede the variable with a dollar sign ($). You can reference variables in shell commands as well as in shell statements.

Explain how to pass data to a shell script.  Shell scripts can use parameters, or arguments, to pass data from the command line to the shell script. Just include the data parameters on the command-line command when starting the shell script. Inside the shell script you can retrieve any command-line parameters by using positional variables. The $0 positional variable is special; it references the shell command used to launch the shell script. Positional variable $1 references the first parameter entered on the command line, variable $2 the second parameter, and so on.

Describe the different types of programming features you can use in shell scripts.  Shell scripts can use conditional statements to evaluate data and make programming decisions based on the outcome of the evaluation. The if and case statements are two conditional statements that you can use for that. The if statement can be paired with the else statement to create a true/false scenario for executing one set of statements if a condition is true, and another set of statements if a condition is false. The case statement allows you to specify a range of possible values that a condition can evaluate to and execute separate sets of statements for each possible value. Besides conditional statements, shell scripts can use loops to iterate through a set of statements multiple times, based on either a series of values or a condition. Finally, shell scripts also support defining functions, which are a set of statements bundled together to be called from anywhere in the shell script as needed.