CHAPTER 5. Using BASH
Exam objectives in this chapter
■ BASH Commands
■ Scheduling Tasks
■ Managing Services
Introduction
The power and versatility of the command line interface (CLI) is a key strength of Linux. It is easily scripted for repetitive tasks, which can then be scheduled and implemented across multiple machines and is straightforward to manage via remote access. BASH is a command interpreter, a way for users to submit instructions to the computer.
BASH Commands
If you have not installed a graphical user interface (GUI), the system will boot you directly into a command shell. Otherwise, you will have to find your systems terminal program, such as Terminal.
Exam Warning
Note that if you do not use a “monospaced” font (where every character is the same width on the screen), columns would not align correctly, and some commands use a variety of colors, making the output invisible if your background matches one of them. A pretty blue background, for example, will render all directories invisible in the default output of the ls' command.
Fast Facts
There are a number of basic commands that you should know:
■ pwd (print working directory) displays the full path of your current directory.
■ ls lists the contents of the current directory or with a path to see the contents of other directories.
■ mkdir creates a directory.
■ rmdir deletes a directory.
■ cd (change directory) changes the current working directory.
■ pushd puts the current directory on a stack.
■ popd retrieves the topmost directory from the stack.
■ touch will create an empty file and can also modify the access and modification times of existing files.
■ cp will copy a file.
■ mv will move (or rename) a file.
■ rm will delete a file.
■ A command prompt often ends with a “$” for a normal user or a “#” if you are running as “root.”
Using File Commands
The commands you type into the command line are programs, and what they do can be controlled based on options you give. The general format of a BASH command is command -option(s) parameter(s).
Crunch Time
Pay careful attention to capitalization as Linux distinguishes between upper and lower case, in commands, options, and file names. For example: ls -s shows file sizes, where ls -S sorts by size. Likewise, FILE1, File1, and file1 are three different files, and ls is a valid command, but LS is not.
The list of options that can be used with ls is large, with the most common ones shown in Table 5.1.
| Option | Description |
|---|---|
| -R | List subdirectories recursively |
| -l | Display detail listing of a directory |
| -a | Do not ignore hidden files beginning with. |
An example line from the output of ls -l is shown below and interpreted in Table 5.2.
drwxr-xr-x 7 chappel users 4096 2009-06-23 21:36 Documents
The type and permissions field is 10 characters long; the left most character is the type of the file, and the next nine bits read, write, or execute permissions for the user, group, and others, respectively, as shown in Table 5.3.
| 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 |
|---|---|---|---|---|---|---|---|---|---|
| File type | User Permissions | Group Permissions | Other Permissions | ||||||
| d | r | w | x | r | w | x | r | w | x |
The first character in the type and permissions field is the file type and can be
■ d (directory)
■ - (regular file)
■ s (socket)
■ p (named pipe)
■ l (symbolic link)
■ c (character device special file)
■ b (block device special file)
The user, group, and other permissions will have one of the following five characters in it:
■ r (read)
■ w (write)
■ x (execute)
■ s (setuid; only found in the execute field)
■ - (in a field means that permission is not set)
The parameters of a command can also alter the results it will give. The most common parameter variations are the wildcards “*” and “+.” These are symbols that can represent multiple patterns; “*” for any number of any character and “?” for just one of any character.
There are a number of commands that can help you to look for files. find is very flexible, requiring a place to start, and what to look for, for example find /home -name file 1 looks for a file named “file1” in the /home directory, then continuing to look in each subdirectory.
locate creates a database of file names that it searches through making it faster. It can only find information in its database, so changes since the last automatically scheduled database update would not be found, and it can only search on file names. The slocate flavor of locate works the same way, but it adds additional security to prevent users from searching through files they do not have access to.
whereis is intended for searching for commands and files related to them – source, binary, configurations, and help files. It does not look in user directories at all. which looks through your path and tells you what program you will run if you type a given command.
File Types
A proper understanding of linked files requires a little technical background information. Directories contain the names of the files that reside in them, and pointers to an inode for each file. The inode stores the metadata for each file – the owner, size, access rights, time of last modification, and where the data contained in each file is physically located. The extra layer of abstraction provided by the inode means that it is possible to have two different names that point to exactly the same file information. This is called a “hardlink” and is created as follows: ln file_name hard_link_name
The hardlink name is just as valid of a name as the original file name, and all hard links must be deleted before the space occupied by the file is freed. The nature of hard links does not permit hard links to directories or to files residing on separate drive volumes.
The soft link is an actual file, with its own inode number and contains the path to the object the file links to. Soft links are allowed to point to directories and to files on other file systems, but because they are independent files it is possible to delete the target of the soft link and leave the link file “broken,” pointing to a nonexistent file. The ls -l command shows symbolic links with an “l” at the very first position in the row and with an arrow (→) pointing to the target file. Symbolic links are created as follows: ln -s file_name soft_link_name
Device Files
Anything capable of moving information in or out of your Linux system has a device file in the /dev subdirectory. This way of thinking about devices makes it easy for user programs to send or receive information; they only need to write (or read) from the appropriate file.
Block special device files move data in and out of hardware connected to the system in large chunks, and use buffers to improve efficiency, such as hard drives, USB, and tape drives. Character device files move data a single character at a time and are unbuffered. Note that not every file in the /dev directory corresponds to a currently connected device; most are simply placeholders waiting for a new device to point to.
Named pipes FIFO (first-in first-out) are buffers that are used for communications between programs running within the computer. One program opens the pipe to write, and the other to read, and data gets transferred between them.
Special files are created using the mknod command, shown for character, block, and pipe files, respectively:
mknod new_file_name c [major_device_number] [minor_device_number]
mknod new_file_name b [major_device_number] [minor_device_number]
mknod new_pipe_name p
Normally, special files are created manually only when device drivers are installed manually, which should include detailed instructions with additional information.
Testing Files
Files frequently have an extension to help indicate what they contain, which may not be so obvious, and file can sort through things with a syntax file [filename]. An example output is shown in Figure 5.1.
test evaluates expressions, as in test 1 -gt 2 and the answer is returned as an exit value that can be checked to make decisions in a script. The power of the test command really shows some more interesting options, such as test -e [filename] to check if a file exists to be sure not to overwrite it, and test -d [directory_name] to see if a directory exists.
The ls -F command lists files with various file types tagged for easier recognition. While the file and test commands are more useful within scripts, ls -F is more helpful for humans and classifies the file with a special character denoting what it is, such as / for directory, @ for a link file, and * for an executable file.
Editing Files Using vi
A good text editor is called vi and the enhanced version is called vim. The key to dealing with vi is understanding that it has three modes: Command, Ex, and Edit. When first starting vi, you will be in Ex mode, which allows you to move around in the file you are editing and to perform copy and paste commands and other advanced features like recording and replaying macros. To actually enter text, you will need to shift into Edit mode by typing one of the edit keys, and then go back to Ex mode by pressing the Esc key. Command mode is entered from Ex mode by typing colon (:) and is used to enter save files, enter a filename or text to be searched for. You can go back to Ex mode by pressing Esc again.
To transition to Edit mode, use the i key to insert text in front of the current cursor position, or o to insert text onto a new line below the line the cursor is on. Once in Edit mode you can type text as you normally would. Press the Esc key to get back to Ex mode.
When you are finished moving and editing you can press the colon key (:) to go to command mode. From command mode you can search and replace text, save the file, and exit from vi.
Managing Processes
Every program, utility, or bit of code waiting in the background on your Linux machine, is called a process, and it is automatically handled by the kernel. Users can monitor and manage their own processes; system administrators have access to nearly all processes.
PS
ps will show you what programs are being run by your user in the terminal window it is executed in, which is usually just bash and the ps command itself. ps -A will show all the processes currently recognized by the machine. The output from the ps command is formatted into four columns. The first column is the process identification number or PID, which the system uses to uniquely identify each process. The next column is where the process is running – it's “control terminal,” which determines where the program's input and output should go. Next is the amount of time the processor has spent on the process. The last is the “human readable” name of the process.
Some other useful options include the following:
■ -u or --user (username or user ID) will show processes associated with the given user.
■ -C (command_name) will show processes that have the given command name.
■ p, -p, or -pid (process_id) will show processes with the listed process ID. If ps is only given a number as an argument, it assumes the number is a process ID.
■ -t, -tty (ttylist) will show processes on a given tty interface port. A plain “-” will show processes not associated with a port, and a -t without a tty number will assume you want the processes associated with the current port you are using.
KILL
kill will forcefully end a process using the process id, and killall will terminate a process by name instead of PID.
The kill command works by sending a signal to the process, by default the TERM signal. The syntax for signals is kill -s [signal] PID and common signals are shown in Table 5.4.
Signals can also be used to communicate between processes, or by the kernel to report an event or problem back to a process.
TOP
A more versatile utility is top, which dynamically shows running processes in real time, sorted by the amount of CPU time they are using shown in Figure 5.2.
The upper section of the top display shows a summary of the system, including uptime, current users (each bash session counts as a user), and various load and memory usage statistics. The lower part of the display shows the process ID, owner, amounts of various memory, and CPU resources being consumed by each process. One handy feature is a built-in kill option; to terminate a process, just press k and type in the process ID.
PSTREE
pstree shows the relationship between parent and child processes in a hierarchical tree view and pstree -p will also show process IDs. The init process is the parent of all the other processes and always has a PID of 1. Each child process knows its parent process ID (PPID), which is what pstree uses to trace the relationship between processes. PPID numbers can be viewed with the ps -Al command.
To look at not just CPU but also physical and network drive usage you can use the iostat command as shown in Figure 5.3.
The various CPU-related statistics shown at the top of the report are as follows:
■ %user Percentage of processor time spent on user applications
■ %nice Percentage of processor time spent on user applications running at a lowered priority
■ %system Portion of processor time used by the system
■ %iowait Percentage of time the processor has been idle when the IO system has been busy
■ %steal Related to virtualized systems
■ %idle Idle time
The next section of the report shows statistics for drives attached to the system, and include the following:
■ tps (transfers per second) A transfer is considered a request for data
■ Blk_read/s, Blk_wrtn/s Blocks read and written per second
■ Blk_read, Blk_wrtn Total blocks read and written
Nice
A process has a priority value between –20 and +19 called a niceness value. The lower the value, the faster it runs, and niceness is inherited from the parent process. The ps -Al will show niceness in the NI column. The owner of a process can make it run slower, freeing up resources, and only a system administrator can make it run faster. There are two ways to adjust the niceness of a file: nice and renice. To start a program with an adjusted nice value, type the following:
nice -n 10 find/-name my_file.doc
Once the process is running it can be reset to a specific value with root access and renice like this:
sudo renice 15 [PID]
Leveraging I/O Redirection
There are a number of small useful tools that are meant to work together and I/O redirection is the glue that sticks these handy little programs together. These console programs make use of three streams for communicating, as shown in Table 5.5.
| Name | Number | Symbol | Normal Connection |
|---|---|---|---|
| STDIN | 0 | < | Keyboard |
| STDOUT | 1 | > | Console |
| STDERR | 2 | > | Console |
You can use file redirection that allows you to use the output from one program as the input for another, or send it to a file. You can use the contents of a file as the input for a program, or use a succession of programs to create and manipulate information. Common uses of redirection are shown in Table 5.6.
Exam Warning
Be careful when using >, if you redirect your output stream to an existing file it will be overwritten. Use >> to append to the end of an existing file.
Some commands will only accept a certain amount of input at a time, and the solution to this is to use xargs, which will accept the input stream and split it into chunks to pass along a bit at a time and repeatedly running the downstream command until the input stream ends. The following example will move all of Bob's music files into his music directory:
find . -name “*.mp3” -u bob | xargs -i mv {} ./bobs_mp3s
If you put a “;” between two commands, they will run consecutively:
do_this_first; then_do_this; and_finally_this
The “= =” is used to compare the equivalence of two variables, often used in scripting.
Special Devices
The /dev/null file is often called “the bit bucket,” and is most often used to hide unnecessary output from a program embedded in a script. The /dev/random file is the interface to the systems random number generator, and it will produce random numbers to create cryptographic keys often using noise from various device drivers to maintain an “entropy pool” to generate the numbers. Applications that can get by with psuedorandom numbers can use /dev/urandom. The /dev/zero file works like /dev/null if you send data to it, but you can use it for an input stream to create files full of zeros.
Using System Documentation
Linux offers many ways to find additional information for all of its commands. The quickest and most basic assistance comes from using the help option of any given command – usually both -? and –help work.
Exam Warning
Remember that when you are taking the Linux+ exam you will have to know about the Linux documentation systems, but unfortunately you would not have access to them.
Did You Know?
There are five main ways to find information on commands:
■ man [command] will display a brief description of the command from information in /usr/share/man or /usr/share/doc. manpath lists the directory where man looks for information.
■ apropos searches man pages to find what you want, for example, apropos search shows all the entries that have search in their description.
■ whatis displays one-line descriptions of a command from the man page.
■ apropos and whatis use a database created by mandb.
■ The database created by mandb may be handled by makewhatis on some distributions.
■ info is intended as an improved way to manage documentation for all the GNU utilities and provides information in threaded chunks called nodes.
Accessing Kernel and Architecture Information
Linux uses a virtual file system located at /proc to represent activity within the computer, and it allows communication with various kernel and driver components. If you compare ps -A with ls /proc, you will notice that every process has its own subdirectory under /proc. There are also files for system hardware: cat /proc/cpuinfo tells you everything you could ever want to know about your processor and /proc/version knows all about the installed version of Linux. A slightly different way to check on your installed version is uname -a. Note that /proc/version shows the actual distribution name, whereas uname -a will show if you have a 32 or 64 bit processor.
All the settings in the proc/sys subdirectory tree can be adjusted usually using sysctl. Changes are lost when the system reboots, and must be added them to /etc/sysctl.conf to make them permanent. Use sysctl -p to get the system to reread the sysctl.conf file without having to reboot.
Basic Scripting
A script is a plain text file containing a collection of command line programs and options that can be all run as a group. Each line is read one at a time and interpreted, which can make scripts slow. Scripts need to have the execute permission bit set unless the sh command is used, which makes the sh program executable and the script just a parameter.
A bash script file expects to have a first line of #!/bin/bash and this script would like to be run with the bash shell located in the /bin directory.
Using Shell Features
The bash shell provides a history feature that remembers what commands you have previously entered, stored in a hidden file in your home directory: .bash_history and the entire list are viewed by typing history. Typing !! will repeat the last command. When you type history each command line is preceded by a number; typing !<command_number> replays the command corresponding to that number. Another handy command line feature is tab completion. Bash will try and guess what you are attempting to type if you press the Tab key. If you already have enough characters to uniquely identify the next word, bash will just fill it in for you. If you have not typed enough of a word, a second press of the Tab key will display a list of possible matches for you.
Scheduling Tasks
In the following sections, you will learn how to put routine tasks on an automated schedule using cron and schedule ad hoc tasks using atq.
cron (cron allow, cron deny)
For the routine tasks that need to be performed on a regular basis or at predefined intervals, there is a scheduling service cron that lets you put these tasks on a schedule of your own design which runs in the background, checking every minute to see if there is anything scheduled to run. The /etc/cron.allow file lists the ONLY users able to edit cron configuration (crontab) files. If that file is empty the system looks for users in the /etc/cron.deny file, which lists users barred from editing cron files. The root user is not affected by either.
crontab Command Syntax
crontab files (cron table) tells cron what to run and when to run it and is stored for users in /var/spool/cron, with the crontab name matching the username. The administrators' files are kept in /etc/crontab, and there is a /etc/cron.d directory that programs can use to store their own schedule files.
Fast Facts
crontab schedule files are edited using crontab –e with each line having six or seven fields and specifying one command.
■ The first five fields on each line tell which minute, hour, day, month, and weekday to run a command.
■ Next specifies whose user rights to run the command under.
■ Lastly is the command itself.
■ Valid options for the schedule fields are a * (match anything), exact number, two numbers with a dash between them for a range of values, or a list of values separated by commas to match each single value.
■ crontab -l displays the contents of your crontab file.
■ crontab -r deletes it.
atq
If you just want to have something run a bit later, use the at command. It accepts commands from either a file (-f) or standard input, ending with a <ctrl> d. A single Ctrl + d is sufficient if there is nothing else on the line. It takes two Ctrl + d's if there is more text on the line. Use atq to see what jobs are queued up, and their job numbers. To cancel a job, use atrm followed by the job number you want canceled.
Managing Services
As a system administrator, one of your primary jobs is to manage services running on your systems.
Fast Facts
Services, or daemons, are just programs that start and run in the background and provide services for users.
■ Services are started when their assigned runlevel is initiated.
■ Scripts that start each service are in etc/init.d.
■ The scripts can be used manually with parameters of start, stop, or restart.
inetd and xinetd
To conserve resources, some smaller and less frequently used network-based services get bundled together in a super server service, which waits in the background until one of the services is needed and then loads it. This method prevents the service from just sitting there taking up memory until it is actually needed. There are two super server daemons in Linux – inetd and the newer, “extended” xinetd, which adds some features and enhanced security. Changing the services managed by inetd or xinetd is done through GUI tools or manually by editing the appropriate configuration file (/etc/inetd.conf or /etc/xinetd.conf) and restarting the service.
chkconfig
chkconfig is a handy tool for managing which runlevels a service runs in some Linux distributions. It can be used to view or change the runlevels that a service will run in. Type chkconfig to view the status of all services at the current runlevel.
Summary of Exam Objectives
In the BASH and command line tools section, we covered enough to get you started creating, editing, moving, and deleting files and directories, as well as using some of the tools that are available to find files within a directory structure. We also covered using man and info to find more information about command line utilities. We even learned about linking commands together to get even more functionality out of them, and touched on how scripts work.
In the Task Scheduling section, we learned how to get programs to run automatically on a regular schedule with cron or just at a later time with at. In the Managing Services section, we learned how to tell Linux which services to run depending on the current runlevel, and how to start, restart, and stop services using the scripts in /etc/init.d.
1. You are documenting your system, and want a file named user_dirs that contains a current list of all the user home directories. What is a quick way of doing this?
A. echo /home > user_dirs
B. ls /home > user_dirs
C. ls /home >> user_dirs
D. cp /home >> user_dirs
2. You want to check that the NTP process is really running and that your computer clock is not slow. What command will confirm that ntp is running?
A. ps -A | grep ntp
B. ps -C ntp
C. ps ntp
D. /usr/init.d/ntp status
3. You have a script named some_program that needs to start in 30 min, and you decide to try the at command instead of setting the alarm on your watch to remind yourself. What is the correct syntax?
A. at 30 <return> some_program <return> <ctrl>d
B. at now + 30 minutes <return> some_program <return> <ctrl>d
C. at 30 minutes some_program <ctrl>d
D. at now + .5 hours <return> some_program <return> <ctrl>d
4. You are copying a bunch of files from ./temp to ./new_stuff, but accidentally type cp temp/* new-stuff instead of new_stuff. You have been reading up on the command history function and want to use that to reenter the command correctly. What do you type?
A. ! -change new-stuff new_stuff
B. !!s/new-stuff>new_stuff
C. ^-^_
D. history -r new-stuff new_stuff
5. You just made a new script my_new_script and you want to run it. You remember to give an explicit path to it when you execute it by typing ./my_ new_script, but you only get an error saying you do not have permission to run the file. You remember that you need to give yourself execution rights to the file. How do you do that?
A. chmod 744 my_new_script
B. chmod 666 my_new_script
C. chmod u+w my_new_script
D. chmod g+x my_new_script
Answers
1. Correct answers and explanations: B. Answer B is the correct because it will list the contents of the /home directory and that listing will be placed in the user_dirs file. Note that B will overwrite any existing file. If desired, using ls -1 will format the output in a single column.
Incorrect answers and explanations: A, C, and D. Answer A is incorrect because echo will not list the contents of the folder; you will get a file users_dirs that contains the text “/home.” While the command in Answer C will append the directory listing to the end of the user_dirs file, if the information in the file was captured on an earlier date and that information has changed, you will end up a file that has both current information from the most recent execution of the command and obsolete information from the previous execution. Answer D is incorrect because the cp command will attempt to copy the /home folder to a new destination. Because there is no destination listed, an error will be generated on the console (because stderr wasn't redirected) and the file user_dirs will be created but empty.
2. Correct answers and explanations: A. Answer A is correct because it will list all running programs, then search for a line that contains “ntp” and show it on the screen. Note that it will only confirm that the process is running, not if it is synchronized.
Incorrect answers and explanations: B, C, and D. Answer B is incorrect because ps -C ntp will look for an exact match for “ntp.” The actual service is ntpd, so no match will be found. Answer C is incorrect because ps will interpret the ntp as an option, which is invalid. Answer D is incorrect because while the ntp init script supports the status flag which will confirm it is running and show the synchronization status; it is located in /etc/ init.d and not /usr/init.d.
3. Correct answer and explanation: B. Answer B is the correct because the keyword now understands adding time using the + sign and descriptive time units. The return is necessary to start the input for what command(s) to run, and the <ctrl>d is needed on a separate line to end the input.
Incorrect answers and explanations: A, C, and D. Answer A is incorrect because the time parameter is wrong, and no units are given. Answer C is incorrect because the time format is wrong and the script that is to run is not on a separate line. Answer D is incorrect because the time parameter must be given in whole numbers.
4. Correct answer and explanation: C. Answer C is correct because the first ^ identifies text to be searched for and the second ^ identifies the text that will be inserted in place of the found text.
Incorrect answers and explanations: A, B, and D. Answer A is incorrect because a single ! expects a line number or beginning characters from the history file; it does not take options. The syntax in Answer B is incorrect; the “greater than” character should be a fore slash; the proper syntax is !!s/new-stuff/new_stuff. Answer D is incorrect because the history command option -r is not used for replacement.
5. Correct answers and explanations: A. Answer A is correct because it sets the file's permissions to allow read, write and execute for the owner, but only read access for group and everyone else.
Incorrect answers and explanations: B, C, and D. Answer B is incorrect because it sets user, group, and everyone else's rights to read and write, but not execute. Answer C is incorrect because it adds write permission for the file owner, but not the execute permission, which is the one that is required. Answer D is incorrect because it adds the execute permission for the files group. Note that even if you are a member of the files group, if you are the owner you will need user execute rights to execute the file.
..................Content has been hidden....................
You can't read the all page of ebook, please click here login for view all page.