Running Multiple Commands

So far in this book we’ve been entering a single command at the command prompt and viewing the results. One exciting feature of the Linux command line is that you can enter multiple commands on the same command line and Linux will process them all. Just place a semicolon between each command you enter:

$ date ; who
Thu Feb 20 19:20:06 EST 2019
rich     :0           2019-02-20 19:15 (:0)
$

The Linux bash shell runs the first command (date) and displays the output; then it runs the second command (who) and displays the output from that command immediately following the output from the first command. While this may seem trivial, this is the basis of how shell scripts work.

Redirecting Output

Another building block of shell scripting is the ability to store command output. Often, when you run a command, you’d like to save the output for future reference. To help with this, the bash shell provides output redirection.

Output redirection allows us to redirect the output of a command from the monitor to another device, such as a file. This feature comes in handy when you need to log data from a shell script that runs after business hours, so you can see what the shell script did when it ran.

To redirect the output from a command, you use the greater-than symbol (>) after the command and then specify the name of the file that you want to use to capture the redirected output. This is demonstrated in Listing 25.1.

Listing 25.1: Redirecting output to a file

$ date > today.txt
$ cat today.txt
Thu Feb 20 19:21:12 EST 2019
$

The example shown in Listing 25.1 redirects the output of the date command to the file named today.txt. Notice that when you redirect the output of a command, nothing displays on the monitor output. All of the text from the output is now in the file, as shown by using the cat command to display the file contents.

The greater-than output redirection operator automatically creates a new file for the output, even if the file already exists. If you prefer, you can append the output to an existing file by using the double greater-than symbol (>>), as shown in Listing 25.2.

Listing 25.2: Appending command output to a file

$ who >> today.txt
$ cat today.txt
Thu Feb 20 19:21:12 EST 2019
rich     :0           2019-02-20 19:15 (:0)
$

The today.txt file now contains the output from the original date command in Listing 25.1 and the output from the who command ran in Listing 25.2.

Output redirection is a crucial feature in shell scripts. With it, we can generate log files from our scripts, giving us a chance to keep track of things as the script runs in background mode on the Linux system.

Piping Data

While output redirection allows us to redirect command output to a file, piping allows us to redirect the output to another command. The second command uses the redirected output from the first command as input data. This feature comes in handy when using commands that process data, such as the sort command.

The piping symbol is the bar (|) symbol, which usually appears above the backslash key on US keyboards. Listing 25.3 shows an example of using piping.

Listing 25.3: Piping command output to another command

$ ls | sort
Desktop
Documents
Downloads
Music
Pictures
Public
Templates
test.txt
today.txt
Videos
$

The output from the ls command is sent directly to the sort command as input, but behind the scenes. You don’t see the output from the ls command displayed on the monitor; you only see the output from the last command in the pipe line, which in this case is the sort command. There’s no limit on how many commands you can link together with piping.

The Shell Script Format

Placing multiple commands on a single line, by using either the semicolon or piping, is a great way to process data but can still get rather tedious. Each time you want to run the set of commands, you need to type them all at the command prompt.

However, Linux allows us to place multiple commands in a text file and then run the text file as a program from the command line. This is called a shell script because we’re scripting out commands for the Linux shell to run.

Shell script files are plain-text files. To create a shell script file, you just need to use any text editor that you’re comfortable with. If you’re working from a KDE-based graphical desktop, you can use the KWrite program, or if you’re working from a GNOME-based graphical desktop, you can use the GEdit program.

If you’re working directly in a command-line environment, you still have some options. Many Linux distributions include either the pico or nano editor to provide a graphical editor environment by using ASCII control characters to create a full-screen editing window.

If your Linux distribution doesn’t include either the pico or nano editor, there is still one last resort: the vi editor. The vi editor is a text-based editor that uses simple single-letter commands. It’s the oldest text editor in the Linux environment, dating back to the early days of Unix, which may be one reason it’s not overly elegant or user-friendly.

Once you’ve chosen your text editor, you’re ready to create your shell scripts. First, for your shell script to work you’ll need to follow a specific format for the shell script file. The first line in the file must specify the Linux shell required to run the script. This is written in somewhat of an odd format:

#!/bin/bash

The Linux world calls the combination of the pound sign and the exclamation symbol (#!) the shebang. It signals to the operating system which shell to use to run the shell script. Most Linux distributions support multiple Linux shells, but the most common is the bash shell. You can run shell scripts written for other shells as long as those shells are installed on the Linux distribution.

After you specify the shell, you’re ready to start listing the commands in your script. You don’t need to enter all of the commands on a single line, Linux allows you to place them on separate lines. Also, the Linux shell assumes each line is a new command in the shell script, so you don’t need to use semicolons to separate the commands. Listing 25.4 shows an example of a simple shell script file.

Listing 25.4: A simple shell script file

$ cat test1.sh
#!/bin/bash
# This script displays the date and who’s logged in
date
who
$

The test1.sh script file shown in Listing 25.4 starts out with the shebang line identifying the bash shell, the standard shell in Linux. The second line in the code shown in Listing 25.4 demonstrates another feature in shell scripts. Lines that start with a pound sign are called comment lines. They allow you to embed comments into the shell script program to help you remember what the code is doing. The shell skips comment lines when processing the shell script. You can place comment lines anywhere in your shell script file after the opening shebang line.

Running the Shell Script

If you just enter a shell script file at the command prompt to run it, you may be a bit disappointed:

$ test1.sh
test1.sh: command not found
$

Unfortunately, the shell doesn’t know where to find the test1.sh command in the virtual directory. The reason for this is the shell uses a special environment variable called PATH to list directories where it looks for commands. If your local HOME folder is not included in the PATH environment variable list of directories, you can’t run the shell script file directly. Instead, you need to use either a relative or an absolute path name to point to the shell script file. The easiest way to do that is by adding the ./ relative path shortcut to the file:

$ ./test1.sh
bash: ./test1.sh: Permission denied
$

Now the shell can find the program file, but there’s still an error message. This time the error is telling us that we don’t have permissions to run the shell script file. A quick look at the shell script file using the ls command with the -l option shows the permissions set for the file:

$ ls -l test1.sh
-rw-r--r-- 1 rich rich 73 Feb 20 19:37 test1.sh
$

By default, the Linux system didn’t give anyone execute permissions to run the file. You can use the chmod command to add that permission for the file owner:

$ chmod u+x test1.sh
$ ls -l test1.sh
-rwxr--r-- 1 rich rich 73 Feb 20 19:37 test1.sh
$

The u+x option adds execute privileges to the owner of the file. You should now be able to run the shell script file and see the output:

$ ./test1.sh
Thu Feb 20 19:48:27 EST 2019
rich     :0           2019-02-20 19:15 (:0)
$

Now that you’ve seen the basics for creating and running shell scripts, the next sections dive into some more advanced features you can add to make fancier shell scripts.

Displaying Messages

When you string commands together in a shell script file, the output may be somewhat confusing to look at. It would help to be able to customize the output by separating it and adding our own text between the output from each listed command.

The echo command allows you to display text messages from the command line. When used at the command line, it’s not too exciting:

$ echo This is a test
This is a test
$

But now you have the ability to insert messages anywhere in the output from the shell script file. Listing 25.5 demonstrates how this is done.

Listing 25.5: Using the echo statement in a script

$ cat test1.sh
#!/bin/bash
# This script displays the date and who’s logged in
echo The current date and time is:
date
echo
echo "Let’s see who’s logged into the system:"
who
$ ./test1.sh
The current date and time is:
Thu Feb 20 19:55:44 EST 2019

Let’s see who’s logged into the system:
rich     :0           2019-02-20 19:15 (:0)
$

The shell script shown in Listing 25.5 adds three echo commands to the test1.sh script. Notice that the first echo command doesn’t use any quotes, but the third one does. The reason for that is the text output from the third echo command contains single quotes. The single quote is also a metacharacter in the shell, which will confuse the echo command, so you need to place double quotes around the text. Also notice that the second echo command doesn’t have any text on the line. That outputs a blank line, which is useful when you want to separate output from multiple commands.

Using Variables

Part of programming is the ability to temporarily store data to use later in the program. You do that by using variables.

Variables allow you to set aside locations in memory to temporarily store information and then recall that information later in the script by referencing the variable name.

There are two types of variables available in the Linux shell. The following sections explain how to use both types in your shell scripts.

$ set
BASH=/bin/bash
BASHOPTS=checkwinsize:cmdhist:complete_fullquote:expand_aliases:extglob:extquote
:force_fignore:histappend:interactive_comments:progcomp:promptvars:sourcepath
BASH_ALIASES=()
BASH_ARGC=()
BASH_ARGV=()
BASH_CMDS=()
BASH_COMPLETION_VERSINFO=([0]="2" [1]="8")
BASH_LINENO=()
BASH_SOURCE=()
BASH_VERSINFO=([0]="4" [1]="4" [2]="19" [3]="1" [4]="release" [5]="x86_64-pc-lin
ux-gnu")
BASH_VERSION=’4.4.19(1)-release’
CLUTTER_IM_MODULE=xim
COLORTERM=truecolor
COLUMNS=80
DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/1000/bus
DESKTOP_SESSION=ubuntu
DIRSTACK=()
DISPLAY=:0
EUID=1000
GDMSESSION=ubuntu
...

$ cat test2.sh
#!/bin/bash
# display user information from the system.
echo User info for userid: $USER
echo UID: $UID
echo HOME: $HOME
$

$ chmod u+x test2.sh
$ ./test2.sh
User info for userid: rich
UID: 1000
HOME: /home/rich
$

var1=10
var2=23.45
var3=testing
var4="Still more testing"

$ cat test3.sh
#!/bin/bash
# testing variables
days=10
guest=Katie
echo $guest checked in $days days ago
$

$ chmod u+x test3.sh
$ ./test3.sh
Katie checked in 10 days ago
$

Command-Line Arguments

One of the most versatile features of shell scripts is the ability to pass data into the script when you run it. This allows you to customize the script with new data each time you run it.

One method of passing data into a shell script is to use command-line arguments. Command-line arguments are data you include on the command line when you run the command. Just start listing them after the command, separating each data value with a space, in this format:

command argument1 argument2 ...

You retrieve the values in your shell script code using special numeric positional variables. Use the variable $1 to retrieve the first command-line argument, $2 the second argument, and so on. Listing 25.9 shows how to use positional variables in your shell script.

Listing 25.9: Using command-line arguments in a shell script

$ cat test4.sh
#!/bin/bash
# Testing command line arguments
echo $1 checked in $2 days ago
$ chmod u+x test4.sh
$ ./test4.sh Barbara 4
Barbara checked in 4 days ago
$ ./test4.sh Jessica 5
Jessica checked in 5 days ago
$

The test4.sh shell script uses two command-line arguments. The $1 variable holds the name of the person, and the $2 variable holds the number of days ago they checked in. When you run the test4.sh shell script, be sure to include both data values in the command line. The shell won’t produce an error message if a positional variable doesn’t exist; you just won’t get the results you expected:

$ ./test4.sh rich
rich checked in days ago
$

It’s up to you to check if the positional variable exists within your program code. We’ll explore how to do that later when we discuss logic statements.

The Exit Status

When a shell script ends, it returns an exit status to the parent shell that launched it. The exit status tells us if the shell script completed successfully or not.

Linux provides us with the special $? variable, which holds the exit status value from the last command that executed. To check the exit status of a command, you must view the $? variable immediately after the command ends. It changes values according to the exit status of the last command executed by the shell:

$ who
rich     :0           2019-02-20 23:16 (:0)
$ echo $?
0
$

By convention, the exit status of a command that successfully completes is 0. If a command completes with an error, then a positive integer value appears as the exit status.

You can change the exit status of your shell scripts by using the exit command. Just specify the exit status value you want in the exit command:

$ /bin/bash
$ exit 120
exit
$ echo $?
120
$

In this example we started a new child shell with the /bin/bash command and then used the exit command to exit the child shell with an exit status code of 120. Back in the parent shell, we then displayed the $? variable value to see if it matched what we had set in the exit command. As you write more complicated scripts, you can indicate errors by changing the exit status value. That way, by checking the exit status, you can easily debug your shell scripts.

Command Substitution

Quite possibly one of the most useful features of shell scripts is the ability to store and process data. So far we’ve discussed how to use output redirection to store output from a command to a file and piping to redirect the output of a command to another command. There’s another technique, however, that can give you more flexibility in storing and using data in your scripts.

Command substitution allows you to assign the output of a command to a user variable in the shell script. After the output is stored in a variable, you can use standard Linux string manipulation commands (such as sort or grep) to manipulate the data before displaying it.

To redirect the output of a command to a variable, you need to use one of two command substitution formats:

• Placing backticks (ˋ) around the command
• Using the command within the $() function

Both methods produce the same result—redirecting the output from the command into a user variable. Listing 25.10 demonstrates using both methods.

Listing 25.10: Demonstrating command substitution

$ var1=ˋdateˋ
$ echo $var1
Fri Feb 21 18:05:38 EST 2019
$ var2=$(who)
$ echo $var2
rich :0 2019-02-21 17:56 (:0)
$

The output from the command substitutions is stored in the appropriate variables. You can then use those variables anywhere in your script program as a standard string value.

Performing Math

Eventually you’ll want to do more than just manipulate text strings in your shell scripts. The world revolves around numbers, and at some point you’ll probably need to do some mathematical operations with your data. Unfortunately, this is one place where the bash shell shows its age. The mathematical features in the bash shell aren’t quite as fancy as the features found in newer shells, such as the Z shell. However, there are a couple of ways to use simple mathematical functions in bash shell scripts.

To include mathematical expressions in your shell scripts, you use a special format. This format places the equation within the $[] characters:

result=$[ 25 * 5 ]

You can perform lots of different mathematical operations on data using this method, but there is a limitation. The $[] format allows you to use only integers; it doesn’t support floating-point values.

If you need to do floating-point calculations, things get considerably more complicated in the bash shell. One solution is to use the bc command-line calculator program. The bc calculator is a tool in Linux that can perform floating-point arithmetic:

$ bc
bc 1.07.1
Copyright 1991-1994, 1997, 1998, 2000, 2004, 2006, 2008, 2012-2017 Free
 Software Foundation, Inc.
This is free software with ABSOLUTELY NO WARRANTY.
For details type ˋwarranty’.
12 * 5.4
64.8
3.156 * (3 + 5)
25.248
quit
$

Unfortunately, the bc calculator has some limitations of its own. The floating-point arithmetic is controlled by a built-in variable called scale. You must set this variable to the desired number of decimal places you want in your answers or you won’t get what you were looking for:

$ bc -q
3.44 / 5
0
scale=4
3.44 / 5
.6880
quit
$

To embed a bc calculation into your script, things get a bit complicated. You must use command substitution to capture the output of the calculation into a variable, but there’s a twist. The basic format you need to use is as follows:

variable=$(echo "options; expression" | bc)

The first parameter, options, allows us to set the bc variables, such as the scale variable. The expression parameter defines the mathematical expression to evaluate using bc. While this looks pretty odd, it works:

$ var1=$(echo "scale=4; 3.44 / 5" | bc)
$ echo $var1
.6880
$

This is not ideal, but it works for small projects. If you have a larger programming project that requires lots of calculations, we’d suggest looking into the Z shell. It supports lots of advanced mathematical functions and features.

Logic Statements

So far all of the shell scripts presented process commands in a linear fashion—one command after another. However, not all programming is linear. There are times when you’d like your program to test for certain conditions, such as if a file exists or if a mathematical expression is 0, and perform different commands based on the results of the test. For that, the bash shell provides logic statements.

Logic statements allow us to test for a specific condition and then branch to different sections of code based on whether the condition evaluates to a True or False logical value. There are a couple of different ways to implement logic statements in bash scripts.

if [ condition ]
then
    commands
fi

$ cat test5.sh
#!/bin/bash
# testing the if condition
if [ $1 -eq $2 ]
then
   echo "Both values are equal!"
   exit
fi

if [ $1 -gt $2 ]
then
   echo "The first value is greater than the second"
   exit
fi

if [ $1 -lt $2 ]
then
   echo "The first value is less than the second"
   exit
fi
$

$ chmod u+x test5.sh
$ ./test5.sh 10 5
The first value is greater than the second
$

case variable in pattern1) commands1;; pattern2 | pattern3) commands2;;
*) default commands;;
esac

$ cat test6.sh
#!/bin/bash
# using the case statement

case $USER in
rich | barbara)
   echo "Welcome, $USER"
   echo "Please enjoy your visit";;
testing)
   echo "Special testing account";;
jessica)
   echo "Don’t forget to log off when you’re done";;
*)
   echo "Sorry, you’re not allowed here";;
esac
$ chmod u+x test6.sh
$ ./test6.sh
Welcome, rich
Please enjoy your visit
$

Loops

When you’re writing scripts, you’ll often find yourself in a situation where it would come in handy to repeat the same commands multiple times, such as applying a command against all of the files in a directory. The bash shell provides some basic looping commands to accommodate that.

for variable in series ; do
   commands
done

$ cat test7.sh
#!/bin/bash
# iterate through the files in the Home folder
for file in $(ls | sort) ; do
   if [ -d $file ]
   then
      echo "$file is a directory"
   fi
   if [ -f $file ]
   then
      echo "$file is a file"
   fi
done
$

$ ./test7.sh
Desktop is a directory
Documents is a directory
Downloads is a directory
Music is a directory
Pictures is a directory
Public is a directory
Templates is a directory
test1.sh is a file
test2.sh is a file
test3.sh is a file
test4.sh is a file
test5.sh is a file
test6.sh is a file
test7.sh is a file
today.txt is a file
Videos is a directory
$

while [ condition ] ; do
   commands
done

$ cat test8.sh
#!/bin/bash
number=$1
factorial=1
while [ $number -gt 0 ] ; do
   factorial=$[ $factorial * $number ]
   number=$[ $number - 1 ]
done
echo The factorial of $1 is $factorial

$ ./test8.sh 5
The factorial of 5 is 120
$ ./test8.sh 6
The factorial of 6 is 720
$