IN THIS CHAPTER
In the earlier years of Linux, people made a big fuss of the graphical environments that were available—they rushed to tell new users that you really did not need to keep going back to the command line to type things. Now that Linux is more mature, people accept that the command line is still a reality, and a welcome one. Although the GUI does make life easier for day-to-day tasks, your options are limited by what the developers wanted you to have—you cannot bring commands together in new ways, and neither can you use any of the GUI features if, for example, your GUI is broken. It does happen!
In his book The Art of UNIX Programming[*], Eric Raymond wrote a short story that perfectly illustrates the power of the command line versus the GUI. It’s reprinted here with permission, for your reading pleasure:
One evening, Master Foo and Nubi attended a gathering of programmers who had met to learn from each other. One of the programmers asked Nubi to what school he and his master belonged. Upon being told they were followers of the Great Way of UNIX, the programmer grew scornful.
“The command-line tools of UNIX are crude and backward,” he scoffed. “Modern, properly designed operating systems do everything through a graphical user interface.”
Master Foo said nothing, but pointed at the moon. A nearby dog began to bark at the master’s hand.
“I don’t understand you!” said the programmer.
Master Foo remained silent, and pointed at an image of the Buddha. Then he pointed at a window.
“What are you trying to tell me?” asked the programmer.
Master Foo pointed at the programmer’s head. Then he pointed at a rock.
“Why can’t you make yourself clear?” demanded the programmer.
Master Foo frowned thoughtfully, tapped the programmer twice on the nose, and dropped him in a nearby trash can.
As the programmer was attempting to extricate himself from the garbage, the dog wandered over and piddled on him.
At that moment, the programmer achieved enlightenment.
Whimsical as the story is, it illustrates that there are some things that the GUI just does not do well. Enter the command line: It is a powerful and flexible operating environment on Linux, and—if you practice—can actually be quite fun, too!
In this chapter, you will learn how to master the command line so that you are able to perform common tasks through it, and also link commands together to create new command groups. We will also be looking at the two most popular Linux text editors: Vim and Emacs. The command line is also known as the shell, the console, the command prompt, and the CLI (command-line interpreter). For the purposes of this chapter these terms are interchangeable, although there are fine-grained differences among them!
Moving from the GUI to the command line is a conscious choice for most people, although it is increasingly rare that it is an either/or choice. While in X, you can press Ctrl+Alt+F1 at any time to switch to a terminal, but most people tend to run an X terminal application that allows them to have the point-and-click applications and the command-line applications side by side.
Reasons for running the shell include
You want to chain two or more commands together.
You want to use a command or parameter available only on the shell.
You are working on a text-only system.
You have used it for a long time and feel comfortable there.
Chaining two or more commands together is what gives the shell its real power. Hundreds of commands are available and, by combining them in different ways, you get hundreds of new commands. Some of the shell commands are available through the GUI, but these commands usually have only a small subset of their parameters available, which limits what you are able to do with them.
Working from a text-only system encapsulates both working locally with a broken GUI and also connecting to a remote, text-only system. If your Linux server is experiencing problems, the last thing you want to do is load it down with a GUI connection—working in text mode is faster and more efficient.
The last use is the most common: People use the shell just because it is familiar to them, with many people even using the shell to start GUI applications just because it saves them taking their hands off the keyboard for a moment! This is not a bad thing; it provides fluency and ease with the system and is a perfectly valid way of working.
It is impossible to know how many commands the average shell citizen uses, but if we had to guess, we would place it at about 25:
cat
—. Prints the contents of a file
cd
—. Changes directories
chmod
—. Changes file access permissions
cp
—. Copies files
du
—. Prints disk usage
emacs
—. Text editor
find
—. Finds files by searching
gcc
—. The C/C++/Fortran compiler
grep
—. Searches for a string in input
less
—. The filter for paging through output
ln
—. Creates links between files
locate
—. Finds files from an index
ls
—. Lists files in the current directory
make
—. Compiles and installs programs
man
—. The manual page reader
mkdir
—. Makes directories
mv
—. Moves files
ps
—. Lists processes
rm
—. Deletes files and directories
ssh
—. Connects to other machines
tail
—. Prints the last lines of a file
vim
—. A text editor
which
—. Prints the location of a command
xargs
—. Executes commands from its input
Of course, many other commands are available to you and are used fairly often—diff
, nmap
, ping
, su
, uptime
, who
, and so on—but if you are able to understand the 25 listed here, you will have sufficient skill to concoct your own command combinations.
Note that we say understand the commands—not know all their possible parameters and usages. This is because several of the commands, although commonly used, are used only in any complex manner by people with specific needs. gcc
and make
are good examples of this: Unless you plan to become a programmer, you need not worry about these beyond just typing make
and make install
now and then. If you want to learn more about these two commands, refer to Chapter 33, “C/C++ Programming Tools for Fedora.”
Similarly, both Emacs and Vim are text editors that have a text-based interface all their own, and SSH has already been covered in detail in Chapter 19, “Remote Access with SSH and Telnet.”
What remains are 20 commands, each of which has many parameters to customize what it actually does. Again, we can eliminate much of this because many of the parameters are esoteric and rarely used, and, the few times in your Linux life that you need them, you can just read the manual page!
We will go over these commands one by one, explaining the most common ways to use them. There is one exception to this: The xargs
command requires you to understand how to join commands together, so it is bundled it in the section “Combining Commands.”
Most of these commands have been touched on elsewhere in the book, primarily in the “Using the Shell” section of Chapter 5, “First Steps with Fedora.” The goal of that section was to give you “the least you need to know” to get by, whereas in this chapter each command is treated individually with an aim to giving you the confidence needed to be able to mix them together to create your own commands.
Many of Fedora’s shell commands manipulate text strings, so if you want to be able to feed them the contents of files, you need to be able to output those files as text. Enter the cat
command, which prints the contents of any files you pass to it. Its most basic use is like this:
cat myfile.txt
That prints the contents of myfile.txt
. For this usage there are two extra parameters that are often used: -n
numbers the lines in the output, and -s
(“squeeze”) prints a maximum of one blank line at a time. That is, if your file has 1 line of text, 10 blank lines, 1 line of text, 10 blank lines, and so on, -s
shows the first line of text, a single blank line, the next line of text, a single blank line, and so forth. When you combine -s
and -n
, cat
numbers only the lines that are printed—the 10 blank lines shown as one will count as 1 line for numbering.
This command prints information about your CPU, stripping out multiple blank lines and numbering the output:
cat -sn /proc/cpuinfo
You can also use cat
to print the contents of several files at once, like this:
cat -s myfile.txt myotherfile.txt
In that command, cat
merges myfile.txt
and myotherfile.txt
on the output, stripping out multiple blank lines. The important thing is that cat
does not distinguish between the files in the output—there are no filenames printed, and no extra breaks between the 2. This allows you to treat the 2 as 1 or, by adding more files to the command line, to treat 20 files as 1.
Changing directories is surely something that has no options, right? Not so. cd
is actually more flexible than most people realize. Unlike most of the other commands here, cd
is not a command in itself—it is built in to bash
(or whichever shell interpreter you are using), but it is still used like a command.
The most basic usage of cd
is this:
cd somedir
That command looks in the current directory for the somedir
subdirectory, and then moves you into it. You can also specify an exact location for a directory, like this:
cd /home/paul/stuff/somedir
The first part of cd
’s magic lies in the characters (-
and ~
, a dash and a tilde). The first means “switch to my previous directory,” and the second means “switch to my home directory.” This conversation with cd
shows this in action:
[paul@caitlin ~]$ cd /usr/local [paul@caitlin local]$ cd bin [paul@caitlin bin]$ cd - /usr/local [paul@caitlin local]$ cd ~ [paul@caitlin ~]$
In the first line, we change to /usr/local
and get no output from the command. In the second line, we change to bin
, which is a subdirectory of /usr/local
. Next, cd -
is used to change back to the previous directory. This time bash
prints the name of the previous directory so we know where we are. Finally, cd ~
is used to change back to our home directory, although if you want to save an extra few keystrokes, just typing cd
by itself is equivalent to cd ~
.
The second part of cd
’s magic is its capability to look for directories in predefined locations. When you specify an absolute path to a directory (that is, one starting with a /
), cd
always switches to that exact location. However, if you specify a relative subdirectory—for example, cd subdir
—you can tell cd
where you would like that to be relative to. This is accomplished with the CDPATH
environment variable. If this variable is not set, cd
always uses the current directory as the base; however, you can set it to any number of other directories.
The next example shows a test of this. It starts in /home/paul/empty
, an empty directory, and the lines are numbered for later reference:
1 [paul@caitlin empty]$ pwd 2 /home/paul/empty 3 [paul@caitlin empty]$ ls 4 [paul@caitlin empty]$ mkdir local 5 [paul@caitlin empty]$ ls 6 local 7 [paul@caitlin empty]$ cd local 8 [paul@caitlin local]$ cd .. 9 [paul@caitlin empty]$ export CDPATH=/usr 10 [paul@caitlin empty]$ cd local 11 /usr/local 12 [paul@caitlin empty]$ cd - 13 /home/paul/empty 14 [paul@caitlin empty]$ export CDPATH=.:/usr 15 [paul@caitlin empty]$ cd local 16 /home/paul/empty/local 17 [paul@caitlin local]$
Lines 1–3 show that we are in /home/paul/empty
and that it is indeed empty—ls
had no output. Lines 4–6 show the local
subdirectory being made so that /home/paul/empty/local
exists. Lines 7 and 8 show you can cd
into /home/paul/empty/local
and back out again.
In line 9, CDPATH
is set to /usr
. This was chosen because Fedora has the directory /usr/local
, which means our current directory (/home/paul/empty
) and our CDPATH
directory (/usr
) both have a local
subdirectory. In line 10, while in the /home/paul/empty
directory, we use cd local
. This time, bash
switches us to /usr/local
and even prints the new directory to ensure that we know what it has done.
Lines 12 and 13 move us back to the previous directory, /home/paul/empty
. In line 14, CDPATH
is set to be .:/usr
. The :
is the directory separator, so this means bash
should look first in the current directory, .
, and then in the /usr
directory. In line 15, cd local
is issued again, this time moving to /home/paul/empty/local
. Note that bash
has still printed the new directory—it does that whenever it looks up a directory in CDPATH
.
What you learned about chmod
in Chapter 5 can be greatly extended through one simple parameter: -c
. This instructs chmod
to print a list of all the changes it made as part of its operation, which means we can capture the output and use it for other purposes. For example:
[paul@caitlin tmp]$ chmod -c 600 * mode of `1.txt' changed to 0600 (rw-------) mode of `2.txt' changed to 0600 (rw-------) mode of `3.txt' changed to 0600 (rw-------) [paul@caitlin tmp]$ chmod -c 600 * [paul@caitlin tmp]$
There the chmod
command is issued with -c
, and you can see it has output the result of the operation: Three files were changed to rw-------
(read and write by user only). However, when the command is issued again, no output is returned. This is because -c
prints only the changes that it made. Files that already match the permissions you are setting are left unchanged and therefore are not printed.
There are two other parameters of interest: --reference
and -R
. The first allows you to specify a file to use as a template for permissions rather than specifying permissions yourself. For example, if you want all files in the current directory to have the same permissions as the file /home/paul/myfile.txt
, you would use this:
chmod --reference /home/paul/myfile.txt *
You can use -R
to enable recursive operation, which means you can use it to chmod
a directory and it changes the permissions of that directory as well as all files and subdirectories under that directory. You could use chmod -R 600 /home
to change every file and directory under /home
to become read/write to their owner.
Like mv
, which is covered later, cp
is a command that is easily used and mastered. However, two marvelous parameters rarely see much use (which is a shame!) despite their power. These are --parents
and -u
, the first of which copies the full path of the file into the new directory. The second copies only if the source file is newer than the destination.
Using --parents
requires a little explanation, so here is an example. You have a file, /home/paul/desktop/documents/work/notes.txt
, and want to copy it to your /home/paul/backup folder
. You could just do a normal cp
, but that would give you /home/
paul/backup/notes.txt
, so how would you know where that came from later? If you use --parents
, the file is copied to /home/paul/backup/desktop/documents/work/notes.txt
.
The -u
parameter is perfect for synchronizing two directories because it allows you to run a command like cp -Ru myfiles myotherfiles
and have cp
recopy only files that have changed. The -R
parameter means recursive and allows you to copy directory contents.
The du
command prints the size of each file and directory that is inside the current directory. Its most basic usage is as easy as it gets:
du
That query outputs a long list of directories and how much space their files take up. You can modify that with the -a
parameter, which instructs du
to print the size of individual files as well as directories. Another useful parameter is -h
, which makes du
use human-readable sizes like 18M (18MB) rather than 17532 (the same number in bytes, unrounded). The final useful basic option is -c
, which prints the total size of files.
So, using du
we can get a printout of the size of each file in our home directory, in human-readable format, and with a summary at the end, like this:
du -ahc /home/paul
Two advanced parameters deal with filenames you want excluded from your count. The first is --exclude
, which allows you to specify a pattern that should be used to exclude files. This pattern is a standard shell file-matching pattern as opposed to a regular expression, which means you can use ?
to match a single character or *
to match 0 or many characters. You can specify multiple --exclude
parameters to exclude several patterns. For example:
du --exclude="*.xml" --exclude="*.xsl"
Of course, typing numerous --exclude
parameters repeatedly is a waste of time, so you can use -X
to specify a file that has the list of patterns you want excluded. The file should look like this:
*.xml *.xsl
That is, each pattern you want excluded should be on a line by itself. If that file were called xml_exclude.txt
, we could use it in place of the previous example like this:
du -X xml_exclude.txt
You can make your exclusion file as long as you need, or you can just specify multiple -X
parameters.
The find
command is one of the darkest and least understood areas of Linux, but it is also one of the most powerful. Admittedly, the find
command does not help itself by using X-style parameters. The UNIX standard is -c
, -s
, and so on, whereas the GNU standard is --dosomething
, --mooby
, and so forth. X-style parameters merge the two by having words preceded by only one dash.
However, the biggest problem with find
is that it has more options than most people can remember—it truly is capable of doing most things you could want. The most basic usage is this:
find -name "*.txt"
That query searches the current directory and all subdirectories for files that end in .txt
. The previous search finds files ending in .txt
but not .TXT
, .Txt
, or other case variations. To search without case sensitivity, use -iname
instead of -name
. You can optionally specify where the search should start before the -name
parameter, like this:
find /home -name "*.txt"
Another useful test is -size
, which lets you specify how big the files should be to match. You can specify your size in kilobytes and optionally also use +
or -
to specify greater than or less than. For example:
find /home -name "*.txt" -size 100k find /home -name "*.txt" -size +100k find /home -name "*.txt" -size -100k
The first brings up files of exactly 100KB, the second only files greater than 100KB, and the last only files less than 100KB.
Moving on, the -user
option enables you to specify the user that owns the files you are looking for. So, to search for all files in /home
that end with .txt
, are under 100KB, and are owned by user paul
, you would use this:
find /home -name "*.txt" -size -100k -user paul
You can flip any of the conditions by specifying -not
before them. For example, you can add a -not
before -user paul
to find matching files owned by everyone but paul
:
find /home -name "*.txt" -size -100k -not -user paul
You can add as many -not
parameters as you need, even using -not -not
to cancel each other out! (Yes, that is pointless.) Keep in mind, though, that -not -size -100k
is essentially equivalent to -size +100k
, with the exception that the former will match files of exactly 100KB whereas the latter will not.
You can use -perm
to specify which permissions a file should have for it to be matched. This is tricky, so read carefully. The permissions are specified in the same way as with the chmod
command: u
for user, g
for group, o
for others, r
for read, w
for write, and x
for execute. However, before you give the permissions, you need to specify a plus, a minus, or a blank space. If you specify neither a plus nor a minus, the files must exactly match the mode you give. If you specify -
, the files must match all the modes you specify. If you specify +
, the files must match any the modes you specify. Confused yet?
The confusion can be cleared up with some examples. This next command finds all files that have permission o=r
(readable for other users). Notice that if you remove the -name
parameter, it is equivalent to *
because all filenames are matched.
find /home -perm -o=r
Any files that have o=r
set are returned from that query. Those files also might have u=rw
and other permissions, but as long as they have o=r
, they will match. This next query matches all files that have o=rw
set:
find /home -perm -o=rw
However, that query does not match files that are o=r
or o=w
. To be matched, a file must be readable and writeable by other users. If you want to match readable or writeable (or both), you need to use +
, like this:
find /home -perm +o=rw
Similarly, this next query matches files that are only readable by user, group, and others:
find /home -perm -ugo=r
Whereas this query matches files as long as they are readable by the user, or by the group, or by others, or by any combination of the three:
find /home -perm +ugo=r
If you use neither +
nor -
, you are specifying the exact permissions to search for. For example, the next query searches for files that are readable by user, group, and others but not writeable or executable by anyone:
find /home -perm ugo=r
You can be as specific as you need to be with the permissions. For example, this query finds all files that are readable for the user, group, and others and writeable by the user:
find /home -perm ugo=r,u=w
To find files that are not readable by others, use the -not
condition, like this:
find /home -not -perm +o=r
Now, on to the most advanced aspect of the find command: the -exec
parameter. This enables you to execute an external program each time a match is made, passing in the name of the matched file wherever you want it. This has very specific syntax: Your command and its parameters should follow immediately after -exec
, terminated by ;
. You can insert the filename match at any point using {}
(an opening and a closing brace side by side).
So, you can match all text files on the entire system (that is, searching recursively from /
rather than from /home
as in our previous examples) over 10KB, owned by paul
, that are not readable by other users, and then use chmod
to enable reading, like this:
find / -name "*.txt" -size +10k -user paul -not -perm +o=r -exec chmod o+r {} ;
When you type your own -exec
parameters, be sure to include a space before ;
. Otherwise, you might see an error such as missing argument to `-exec'
.
Do you see now why some people think the find
command is scary? Many people learn just enough about find
to be able to use it in a very basic way, but hopefully you will see how much it can do if you give it a chance.
The grep
command, like find
, is an incredibly powerful search tool in the right hands. Unlike find
, though, grep
processes any text, whether in files, or just in standard input.
The basic usage of grep
is this:
grep "some text" *
That query searches all files in the current directory (but not subdirectories) for the string some text
and prints matching lines along with the name of the file. To enable recursive searching in subdirectories, use the -r
parameter, like this:
grep -r "some text" *
Each time a string is matched within a file, the filename and the match are printed. If a file contains multiple matches, each of the matches is printed. You can alter this behavior with the -l
parameter (lowercase L), which forces grep
to print the name of each file that contains at least one match, without printing the matching text. If a file contains more than one match, it is still printed only once. Alternatively, the -c
parameter prints each filename that was searched and includes the number of matches at the end, even if there were no matches.
You have a lot of control when specifying the pattern to search for. You can, as previously, specify a simple string like some text
, or you can invert that search by specifying the -v
parameter. For example, this returns all the lines of the file myfile.txt
that do not contain the word hello
:
grep -v "hello" myfile.txt
You can also use regular expressions for your search term. For example, you can search myfile.txt
for all references to cat
, sat
, or mat
with this command:
grep "[cms]at" myfile.txt
Adding the -i
parameter to that removes case sensitivity, matching Cat
, CAT
, MaT
, and so on:
grep -i [cms]at myfile.txt
The output can also be controlled to some extent with the -n
and --color
parameters. The first tells grep
to print the line number for each match, which is where it appears in the source file. The --color
parameter tells grep
to color the search terms in the output, which helps them stand out when among all the other text on the line. You choose which color you want using the GREP_COLOR
environment variable: export GREP_COLOR=36
gives you cyan, and export GREP_COLOR=32
gives you lime green.
This next example uses these two parameters to number and color all matches to the previous command:
grep -in --color [cms]at myfile.txt
Later you will see how important grep
is for piping with other commands.
The less
command enables you to view large amounts of text in a more convenient way than the cat
command. For example, your /etc/passwd
file is probably more than a screen long, so if you run cat /etc/passwd
, you are not able to see what the lines at the top were. Using less /etc/passwd
enables you to use the cursor keys to scroll up and down the output freely. Type q to quit and return to the shell.
On the surface, less
sounds like a very easy command; however, it has the infamy of being one of the few Linux commands that have a parameter for every letter of the alphabet. That is, -a
does something, -b
does something else, -c
, -d
, -e
, ..., -x
, -y
, -z
—they all do things, with some letters even differentiating between upper- and lowercase. Furthermore, these are only the parameters used to invoke less
. After you are viewing your text, even more commands are available to you. Make no mistake—less
is a complex beast to master.
Input to less
can be divided into two categories: what you type before running less
and what you type while running it. The former category is easy, so we will start there.
We have already discussed how many parameters less
is capable of taking, but they can be distilled down to three that are very useful: -M
, -N
, and +
. Adding -M
(this is different from -m
!) enables verbose prompting in less
. Rather than just printing a colon and a flashing cursor, less
prints the filename, the line numbers being shown, the total number of lines, and the percentage of how far you are through the file. Adding -N
(again, this is different from -n
) enables line numbering.
The last option, +
, allows you to pass a command to less
for it to execute as it starts. To use this, you first need to know the commands available to you in less
, which means we need to move onto the second category of less
input: what you type while less
is running.
The basic navigation keys are the up, down, left, and right cursors; Home and End (for navigating to the start and end of a file); and Page Up and Page Down. Beyond that, the most common command is /
, which initiates a text search. You type what you want to search for and press Enter to have less
find the first match and highlight all subsequent matches. Type /
again and press Enter to have less
jump to the next match. The inverse of that is ?
, which searches backward for text. Type ?
, enter a search string, and press Enter to go to the first previous match of that string, or just use ?
and press Enter to go to the next match preceding the current position. You can use /
and ?
interchangeably by searching for something with /
and then using ?
to go backward in the same search.
Searching with /
and ?
is commonly used with the +
command-line parameter from earlier, which passes less
a command for execution after the file has loaded. For example, you can tell less
to load a file and place the cursor at the first match for the search hello
, like this:
less +/hello myfile.txt
Or, to place the cursor at the last match for the search hello
:
less +?hello myfile.txt
Beyond the cursor keys, the controls primarily involve typing a number and then pressing a key. For example, to go to line 50, type 50g
, or to go to the 75% point of the file, type 75p
. You can also place invisible mark points through the file by pressing m
and then typing a single letter. Later, while in the same less session, you can press '
(a single quote) and then type the letter, and you will move back to the same position. You can set up to 52 marks, named a–z and A–Z.
One clever feature of less
is that you can, at any time, press v
to have your file opened inside your text editor. This defaults to Vim, but you can change that by setting the EDITOR
environment variable to something else.
If you have made it this far, you can already use less
better than most users. You can, at this point, justifiably skip to the next section and consider yourself proficient with less
. However, if you want to be a less
guru, there are two more things to learn: how to view multiple files simultaneously and how to run shell commands.
Like most other file-based commands in Linux, less
can take several files as its parameters. For example:
less -MN 1.txt 2.txt 3.txt
That command loads all three files into less
, starting at 1.txt
. When viewing several files, less
usually tells you which file you are in, as well as numbering them: 1.txt (file 1 of 3)
should be at the bottom of the screen. That said, certain things will make that go away, so you should use -M
anyway.
You can navigate between files by typing a colon and then pressing n
to go to the next file or p
to go to the previous file; these are referred to from now on as :n
and :p
.
You can open another file for viewing by typing :e
and providing a filename. This can be any file you have permission to read, including files outside the local directory. Use the Tab key to complete filenames. Files you open in this way are inserted one place after your current position, so if you are viewing file 1 of 4 and open a new file, the new file is numbered 2 of 5 and is opened for viewing straightaway. To close a file and remove it from the list, use :d
.
Viewing multiple files simultaneously has implications for searching. By default, less
searches within only one file, but it is easy to search within all files. When you type /
or ?
to search, follow it with a *
—you should see EOF-ignore
followed by a search prompt. You can now type a search and it will search in the current file; if nothing is found, it looks in subsequent files until it finds a match. Repeating searches is done by pressing the Esc key and then either n
or N
. The lowercase option repeats the search forward across files, and the uppercase repeats it backward.
The last thing you need to know is that you can get to a shell from the less
and execute
commands. The simplest way to do this is just to type !
and press Enter. This launches a shell and leaves you free to type all the commands you want, as per normal. Type exit
to return to less
. You can also type specific commands by entering them after the exclamation mark, using the special character %
for the current filename. For example, du -h %
prints the size of the current file. Finally, you can use !!
to repeat the previous command.
Linux allows you to create links between files that look and work like normal files for the most part. Moreover, it allows you to make two types of links, known as hard links and symbolic links (symlinks). The difference between the two is crucial, although it might not be obvious at first!
Each filename on your system points to what is known as an inode, which is the absolute location of a file. Linux allows you to point more than one filename to a given inode, and the result is a hard link—two filenames pointing to exactly the same file. Each of these files shares the same contents and attributes. So, if you edit one, the other changes because they are both the same file.
On the other hand, a symlink—sometimes called a soft link—is a redirect to the real file. When a program tries to read from a symlink, it automatically is redirected to what the symlink is pointing at. The fact that symlinks are really just dumb pointers has two advantages: You can link to something that does not exist (and create it later if you want), and you can link to directories.
Both types of links have their uses. Creating a hard link is a great way to back up a file on the same disk. For example, if you delete the file in one location, it still exists untouched in the other location. Symlinks are popular because they allow a file to appear to be in a different location; you could store your website in /var/www/live
and an under-construction holding page in /var/www/construction
. Then you could have Apache point to a symlink /var/www/html
that is redirected to either the live or construction directory depending on what you need.
The shred
command overwrites a file’s contents with random data, allowing for safe deletion. Because this directly affects a file’s contents, rather than just a filename, this means that all filenames hard linked to an inode are affected.
Both types of link are created using the ln
command. By default, the ln
command creates hard links, but you can create symlinks by passing it the -s
parameter. The syntax is ln [-s] <
something
> <
somewhere
>
, for example:
ln -s myfile.txt mylink
That command creates the symlink mylink
that points to myfile.txt
. Remove the -s
to create a hard link. You can verify that your link has been created by running ls -l
. Your symlink should look something like this:
lrwxrwxrwx 1 paul paul 5 Feb 19 12:39 mylink -> myfile.txt
Note how the file properties start with l
(lowercase L) for link and how ls -l
also prints where the link is going. Symlinks are always very small in size; the previous link is five bytes in size. If you created a hard link, it should look like this:
-rw-rw-r 2 paul paul 341 Feb 19 12:39 mylink
This time the file has normal attributes, but the second number is 2 rather than 1. That number is how many hard links point to this file, which is why it is 2 now. The file size is also the same as that of the previous filename because it is the file as opposed to just being a pointer.
Symlinks are used extensively in Linux. Programs that have been superseded, such as sh
, now point to their replacements (in this case, bash
), and library versioning is accomplished through symlinks. For example, applications that link against zlib
load /usr/lib/libz.so
. Internally, however, that is just a symlink that points to the actual zlib
library: /usr/lib/libz.so.1.2.1.2
. This enables multiple versions of libraries to be installed without applications needing to worry about the version name.
When you use the find
command, it searches recursively through each directory each time you request a file. This is slow, as you can imagine. Fortunately, Fedora ships with a cron
job that creates an index of all the files on your system every night. Searching this index is extremely fast, which means that, if the file you are looking for has been around since the last index, this is the preferable way of searching.
To look for a file in your index, use the command locate
followed by the names of the files you want to find, like this:
locate myfile.txt
On a relatively modern computer (1.5GHz or higher), locate
should be able to return all the matching files in less than a second. The trade-off for this speed is lack of flexibility. You can search for matching filenames, but, unlike find
, you cannot search for sizes, owners, access permissions, or other attributes. The one thing you can change is case sensitivity; use the -i
parameter to do a case-insensitive search.
Although Fedora rebuilds the filename index nightly, you can force a rebuild whenever you want by running the command updatedb
as root. This usually takes a few minutes, but when it’s done the new database is immediately available.
The ls
command, like ln
, is one of those you expect to be very straightforward. It lists files, but how many options can it possibly have? In true Linux style, the answer is many, although again you need know only a few to wield great power!
The basic usage is simply ls
, which outputs the files and directories in the current location. You can filter that using normal wildcards, so all these are valid:
ls * ls *.txt ls my*ls *.txt *.xml
Any directories that match these filters are recursed into one level. That is, if you run ls my*
and you have the files myfile1.txt
and myfile2.txt
and a directory mystuff
, the matching files are printed first. Then ls
prints the contents of the mystuff
directory.
The most popular parameters for customizing the output of ls
are
-a
—. Includes hidden files
-h
—. Uses human-readable sizes
-l
—. A lowercase L, it enables long listing
-r
—. Reverse order
-R
—. Recursively lists directories
-s
—. Shows sizes
--sort
—. Sorts the listing
All files that start with a period are hidden in Linux, so that includes the .gnome
directory in your home directory, as well as .bash_history
and the .
and ..
implicit directories that signify the current directory and the parent. By default, ls
does not show these files, but if you run ls -a
, they are shown. You can also use ls -A
to show all the hidden files except .
and ..
.
The -h
parameter has to be combined with the -s
parameter, like this:
ls -sh *.txt
That query outputs the size of each matching file in a human-readable format, such as 108KB or 4.5MB.
Using the -l
parameter enables much more information about your files. Instead of just providing the names of the files, you get output like this:
drwxrwxr-x 24 paul paul 4096 Dec 24 21:33 arch -rw-r--r-- 1 paul paul 18691 Dec 24 21:34 COPYING -rw-r--r-- 1 paul paul 88167 Dec 24 21:35 CREDITS drwxrwxr-x 2 paul paul 4096 Dec 24 21:35 crypto
That output shows four matches and prints a lot of information about each of them. The first row shows the arch
directory; you can tell it is a directory because its file attributes starts with a d. The rwxrwxr-x
following that shows the access permissions, and this has special meanings because it is a directory. Read access for a directory allows users to see the directory contents, write access allows you to create files and subdirectories, and execute access allows you to cd
into the directory. If a user has execute access but not read access, he will be able to cd
into the directory but not list files.
Moving on, the next number on the line is 24
, which also has a special meaning for directories: It is the number of subdirectories, including .
and ..
. After that is paul paul
, which is the name of the user owner and the group owner for the directory. Next are the size and modification time, and finally the directory name itself.
The next line shows the file COPYING
, and most of the numbers have the same meaning, with the exception of the 1
immediately after the access permissions. For directories, this is the number of subdirectories, but for files this is the number of hard links to this file. A 1
in this column means this is the only filename pointing to this inode, so if you delete it, it is gone.
Fedora comes configured with a shortcut command for ls -l
: ll
.
The --sort
parameter allows you to reorder the output from the default alphabetical sorting. You can sort by various things, although the most popular are extension (alphabetically), size (largest first), and time (newest first). To flip the sorting (making size sort by smallest first), use the -r
parameter also. So, this command lists all .ogg
files, sorted smallest to largest:
ls --sort size -r *.ogg
Finally, the -R
parameter recurses through subdirectories. For example, typing ls /etc
lists all the files and subdirectories in /etc
, but ls -R /etc
lists all the files in and subdirectories in /etc
, all the files and subdirectories in /etc/acpi
, all the files and subdirectories in /etc/acpi/actions
, and so on until every subdirectory has been listed.
Time for a much-needed mental break: The man
command is easy to use. Most people use only the topic argument, like this: man gcc
. However, two other commands work closely with man
that are very useful—whatis
and apropos
.
The whatis
command returns a one-line description of another command, which is the same text you get at the top of that command’s man page. For example:
[paul@caitlin ~]$ whatis ls ls (1) - list directory contents
The output there explains what ls
does but also provides the man page section number for the command so that you can query it further.
On the other hand, the apropos
command takes a search string as its parameter and returns all man pages that match the search. For example, apropos mixer
gives this list:
alsamixer (1) - soundcard mixer for ALSA soundcard driver mixer (1) - command-line mixer for ALSA soundcard driver aumix (1) - adjust audio mixer
So, use apropos
to help you find commands and use whatis
to tell you what a command does.
Making directories is as easy as it sounds, although there is one parameter you should be aware of: -p
. If you are in /home/paul
and you want to create the directory /home/paul/audio/sound
, you will get an error like this:
mkdir: cannot create directory `audio/sound`: No such file or directory
At first glance this seems wrong because mkdir
cannot create the directory because it does not exist. What it actually means is that it cannot create the directory sound
because the directory audio
does not exist. This is where the -p
parameter comes in: If you try to make a directory within another directory that does not exist, like the previous, -p
creates the parent directories, too. So:
mkdir -p audio/sound
That first creates the audio
directory and then creates the sound
directory inside it.
This command is one of the easiest around. There are two helpful parameters to mv
: -f
, which overwrites files without asking, and -u
, which moves the source file only if it is newer than the destination file. That is it!
This is the third and last “command that should be simple, but isn’t” that is discussed here. The ps
command lists processes and gives you an extraordinary amount of control over its operation.
The first thing to know is that ps
is typically used with what are known as BSD-style parameters. Back in the section “Finding Files with find
,” we discussed UNIX-style, GNU-style, and X-style parameters (-c
, --dosomething
, and -dosomething
, respectively), but BSD-style parameters are different because they use single letters without a dash.
So, the default use of ps
lists all processes you are running that are attached to the terminal. However, you can ask it to list all your processes attached to any terminal (or indeed no terminal) by adding the x
parameter: ps x
. You can ask it to list all processes for all users with the a
parameter or combine that with x
to list all processes for all users, attached to a terminal or otherwise: ps ax
.
However, both of these are timid compared with the almighty u
option, which enables user-oriented output. In practice, that makes a huge difference because you get important fields such as the username of the owner, how much CPU time and RAM are being used, when the process was started, and more. This outputs a lot of information, so you might want to try adding the f
parameter, which creates a process forest by using ASCII art to connect parent commands with their children. You can combine all the options so far with this command: ps faux
(yes, with a little imagination you spell words with the parameters!).
You can control the order in which the data is returned by using the --sort
parameter. This takes either a +
or a -
(although the +
is default) followed by the field you want to sort by: command
, %cpu
, pid
, and user
are all popular options. If you use the minus sign, the results are reversed. This next command lists all processes, in descending order by CPU usage:
ps aux --sort=-%cpu
There are many other parameters for ps
, including a huge number of options for compatibility with other UNIXes. If you have the time to read the man page, you should give it a try!
The rm
command has only one parameter of interest: --preserve-root
. By now, you should know that issuing rm -rf /
as root will destroy your Linux installation because -r
means recursive and -f
means force (do not prompt for confirmation before deleting). It is possible for a clumsy person to issue this command by accident—not by typing the command on purpose, but by putting a space in the wrong place. For example:
rm -rf /home/paul
That command deletes the home directory of the user paul
. This is not an uncommon command; after you have removed a user and backed up her data, you will probably want to issue something similar. However, if you add an accidental space between the /
and the h
in home
, you get this:
rm -rf / home/paul
This time the command means “delete everything recursively from /
and then delete home/paul
”—quite a different result! You can stop this from happening by using the --preserve-root
parameter, which stops you from catastrophe with this message:
rm: it is dangerous to operate recursively on `/' rm: use --no-preserve-root to override this failsafe.
Of course, no one wants to keep typing --preserve-root
each time they run rm
, so you should add this line to the .bashrc
file in your home directory:
alias rm='rm --preserve-root'
That alias automatically adds --preserve-root
to all calls to rm
in future bash
sessions.
If you want to watch a log file as it is written to, or want to monitor a user’s actions as they are occurring, you need to be able to track log files as they change. In these situations you need the tail
command, which prints the last few lines of a file and updates as new lines are added. This command tells tail
to print the last few lines of /var/log/httpd/access_log
, the Apache hit log:
tail /var/log/httpd/access_log
To get tail
to remain running and update as the file changes, add the -f
parameter (which means “follow”):
tail -f /var/log/httpd/access_log
You can tie the lifespan of a tail
follow to the existence of a process by specifying the --pid
parameter. When you do this, tail
continues to follow the file you asked for until it sees that the process (identified by PID) is no longer running, at which point tail
stops tailing.
If you specify multiple files on the command line, tail
follows both, printing file headers whenever the input source changes. Press Ctrl
+C
to terminate tail
when in follow mode.
The top
command is unusual in this list because the few parameters it takes are rarely, if ever, used. Instead, it has a number of commands you can use while it is running to customize the information it shows you. To get the most from these instructions, open two terminal windows. In the first, run the program yes
and leave it running; in the second, switch to root, and run top
.
The default sort order in top
shows the most CPU-intensive tasks first. The first command there should be the yes
process you just launched from the other terminal, but there should be many others also. First, we want to filter out all the other users and focus on the user running yes
. To do this, press u
and enter the username you used when you ran yes
. When you press Enter, top
filters out processes not being run by that user.
The next step is to kill the process ID of the yes
command, so you need to understand what each of the important fields means:
PID
—. The process ID
User
—. The owner of the process
PR
—. Priority
NI
—. Niceness
Virt
—. Virtual image size in kilobytes
Res
—. Resident size in kilobytes
Shr
—. Shared memory size in kilobytes
S
—. Status
%CPU
—. CPU usage
%Mem
—. Memory usage
Time+
—. CPU time
Command
—. The command being run
Several of these fields are unimportant unless you have a specific problem. The ones we are interested in are PID
, User
, Niceness
, %CPU
, %MEM
, Time+
, and Command
. The Niceness
of a process is how much time the CPU allocates to it compared to everything else on the system: 19 is the lowest, and –19 is the highest.
With the columns explained, you should be able to find the process ID of the errant yes
command launched earlier; it is usually the first number below PID
. Now type k
, enter that process ID, and press Enter. You are prompted for a signal number (the manner in which you want the process killed), with 15 provided as the default. Signal 15 (also known as SIGTERM
, for “terminate”) is a polite way of asking a process to shut down, and all processes that are not wildly out of control should respond to it. Give top
a few seconds to update itself, and the yes
command should be gone. If not, you need to be more forceful: type k
again, enter the PID, and press Enter. When prompted for a signal to send, enter 9
and press Enter to send SIGKILL
, which means “terminate whether you like it or not.”
You can choose the fields to display by pressing f
. A new screen appears that lists all possible fields, along with the letter you need to press to toggle their visibility. Selected fields are marked with an asterisk and have their letter, for example:
* A: PID = Process Id
If you press the a
key, the screen changes to this:
a: PID = Process Id
When you have made your selections, press Enter to return to the normal top
view with your normal column selection.
You can also press F
to select the field you want to use for sorting. This works in the same way as the field selection screen, except that you can select only one field at a time. Again, press Enter to get back to top
after you have made your selection, and it will be updated with the new sorting.
If you press B
, text bolding is enabled. By default, this bolds some of the header bar as well as any programs that are currently running (as opposed to sleeping), but if you press x
you can also enable bolding of the sorted column. You can use y
to toggle bolding of running processes.
The last command to try is r
, which allows you to renice—or adjust the nice value—of a process. You need to enter the PID of the process, press Enter, and enter a new nice value. Keep in mind that 19 is the lowest and –20 is the highest; anything less than 0 is considered high and should be used sparingly.
So far, we have been using commands only individually, and for the large part that is what you will be doing in practice. However, some of the real power of these commands lies in the capability to join them to get exactly what you want. There are some extra little commands that we have not looked at that are often used as glue because they do one very simple thing that enables a more complex process to work.
All the commands we have looked at have printed their information to the screen, but this is often flexible. There are two ways to control where output should go: piping and output redirection. A pipe is a connector between one command’s output and another’s input. Instead of sending its output to your terminal, a command sends that output directly to another command for input. Output redirection works in a similar way to pipes but is usually used for files. We will look at pipes first and then output redirection.
Two of the commands we have looked at so far are ps
and grep
: the process lister and the string matcher. We can combine the two to find out which users are playing Nethack right now:
ps aux | grep nethack
That creates a list of all the processes running right now and sends that list to the grep
command, which filters out all lines that do not contain the word nethack
. Fedora Core Linux allows you to pipe as many commands as you can sanely string together. For example, we could add in the wc
command, which counts the numbers of lines, words, and characters in its input, to count precisely how many times Nethack is being run:
ps aux | grep nethack | wc -l
The -l
(lowercase L) parameter to wc
prints only the line count.
Using pipes in this way is often preferable to using the -exec
parameter to find
, simply because many people consider find
to be a black art and so anything that uses it less frequently is better! This is where the xargs
command comes in: It converts output from one command into arguments for another.
For a good example, consider this mammoth find
command from earlier:
find / -name "*.txt" -size +10k -user paul -not -perm +o=r -exec chmod o+r {} ;
That command searches every directory from /
onward for files matching *.txt
that are greater than 10KB, are owned by user paul
, and do not have read permission for others. Then it executes chmod
on each of the files. It is a complex command, and people who are not familiar with the workings of find
might have problems understanding it. So, what we can do is break up the single command into two parts: a call to find
and a call to xargs
. The most conversion would look like this:
find / -name "*.txt" -size +10k -user paul -not -perm +o=r | xargs chmod o+r
That has eliminated the confusing {} ;
from the end of the command, but it does the same thing, and faster, too. The speed difference between the two is because using -exec
with find causes it to execute chmod
once for each file. However, chmod
accepts many files at a time and, because we are using the same parameter each time, we should take advantage of that. The second command, using xargs
, is called once with all the output from find
, and so saves many command calls. The xargs
command automatically places the input at the end of the line, so the previous command might look something like this:
xargs chmod o+r file1.txt file2.txt file3.txt
Not every command accepts multiple files, though, and if you specify the -l
parameter, xargs
executes its command once for each line in its input. If you want to check what it is doing, use the -p
parameter to have xargs
prompt you before executing each command.
For even more control, the -i
parameter allows you to specify exactly where the matching lines should be placed in your command. This is important if you need the lines to appear before the end of the command or need it to appear more than once. Either way, using the -i
parameter also enables the -l
parameter so that each line is sent to the command individually. This next command finds all files in /home/paul
that are larger than 10,000KB in size (10MB) and copies them to /home/paul/archive
:
find /home/paul -size +10000k | xargs -i cp {} ./home/paul/archive
Using find
with xargs
is a unique case. All too often, people use pipes when parameters would do the job just as well. For example, these two commands are identical:
ps aux --sort=-%cpu | grep -v `whoami` ps -N ux --sort=-%cpu
The former prints all users and processes and then pipes that to grep
, which in turn filters out all lines that contain the output from the program whoami
(our username). So, line one prints all processes being run by other uses, sorted by CPU use. Line two does not specify the a
parameter to ps
, which makes it list only our parameters. It then uses the -N
parameter to flip that, which means it lists everyone but us, without the need for grep
.
The reason people use the former is often just simplicity: Many people know only a handful of parameters to each command, so they can string together two commands simply rather than write one command properly. Unless the command is to be run regularly, this is not a problem. Indeed, the first line would be better because it does not drive people to the manual to find out what ps -N
does!
It is both curious and sad that many Linux veterans have not heard of the screen
command. Curious because they needlessly go to extra effort to replicate what screen
takes in its stride and sad because they are missing a powerful tool that would benefit them greatly.
Picture this scene: You connect to a server via SSH and are working at the remote shell. You need to open another shell window so that you can have the two running side by side; perhaps you want the output from top in one window while typing in another. What do you do? Most people would open another SSH connection, but doing so is both wasteful and unnecessary. screen
is a terminal multiplexer, which is a fancy term for a program that lets you run multiple terminals inside one terminal.
The best way to learn screen
is to try it yourself, so open a console, type screen
, and then press Enter. Your display will blank momentarily and then be replaced with a console; it will look like nothing has changed. Now, let’s do something with that terminal. Run top
and leave it running for the time being. Hold down the Ctrl key and press a
(referred to as Ctrl
+a
from now on); then let go of them both and press c
. Your prompt will clear again, leaving you able to type. Run the uptime
command.
Pop quiz: What happened to the old terminal running top
? It is still running, of course. You can press Ctrl
+a
and then press 0
(zero) to return to it. Press Ctrl
+a
and then press 1
to go back to your uptime
terminal. While you are viewing other terminals, the commands in the other terminals carry on running as normal so that you can multitask.
Many of screen
’s commands are case sensitive, so the lettering used here is very specific: Ctrl
+a
means “press Ctrl and the a key,” but Ctrl
+A
means press “Ctrl and Shift and the a key” so that you get a capital A. Ctrl
+a
+A
means “press Ctrl and the a key, let them go, and then press Shift and the a key.”
You have seen how to create new displays and how to switch between them by number. However, you can also bring up a window list and select windows using your cursor with Ctrl
+a
+"
(that is, press Ctrl and a together, let go, and press the double quotes key [usually Shift and the single quote key]). You will find that the screens you create have the name bash
by default, which is not very descriptive. Select a window and press Ctrl
+a
+A
. You are prompted to enter a name for the current window, and your name is used in the window list.
When you get past window 9, it becomes impossible to switch to windows using Ctrl
+a
and 0
–9
; as soon as you type the 1
of 10
, screen
switches to display 1. The solution is to use either the window list or the quick change option, in which you press Ctrl
+a
+'
(single quote), enter either the screen number or the name you gave it, and then press Enter. You can also change back to the previous window by pressing Ctrl
+a
+Ctrl
+a
. If you work within only a small set of windows, you can use Ctrl
+a
+n
and Ctrl
+a
+p
to move to the next and previous windows, respectively. Of course, if you are changing to and from windows only to see whether something has changed, you are wasting time because screen
can monitor windows for you and report if anything changes. To enable (or disable) monitoring for a window, use Ctrl
+a
+M
; when something happens, screen
flashes a message. If you miss it (the messages disappear when you type something), use Ctrl
+a
+m
to bring up the last message.
Windows close when you kill the main program inside. Using Ctrl
+a
+c
, this window is bash
; type exit
to quit. Alternatively, you can use Ctrl
+a
+K
to kill a window. When all your windows are closed, screen
terminates and prints a screen is terminating
message so that you know you are out.
However, there are two alternatives to quitting: locking and disconnecting. The first, activated with Ctrl
+a
+x
, locks access to your screen data until you enter your system password. The second is the most powerful feature of screen
: You can exit it and do other things for a while and then reconnect later and screen
will pick up where you left off. For example, you could be typing at your desk, disconnect from screen
, and then go home, reconnect, and carry on as if nothing had changed. What’s more, all the programs you ran from screen
carry on running even while screen
is disconnected. It even automatically disconnects for you if someone closes your terminal window while it is in a locked state (with Ctrl
+a
+x
).
To disconnect from screen
, press Ctrl
+a
+d
. You are returned to the prompt from which you launched screen
and can carry on working, close the terminal you had opened, or even log out completely. When you want to reconnect, run the command screen -r
. You can, in the meantime, just run screen
and start a new session without resuming the previous one, but that is not wise if you value your sanity! You can disconnect and reconnect the same screen
session as many times you want, which potentially means you need never lose your session again.
Although this has been a mere taste of screen
, we hope you can see how useful it can be.
http://www.gnu.org/—The website of the GNU project, it contains manuals and downloads for lots of command-line software.
http://www.linuxdevcenter.com/linux/cmd/—A wide selection of Linux commands and explanations of what they do.
http://www.tuxfiles.org/linuxhelp/cli.html—Several short command-line tutorials, courtesy of tuXfiles.
http://www.linuxcommand.org/—Describes itself as “your one-stop command line shop!” It contains a wealth of useful information about the console.
http://www.tripwire.org/—Information and download links for the open source version of Tripwire.
Understanding the way UNIX works at the nuts-and-bolts level can be both challenging and rewarding, and there are several good books that will help guide you on your way. Perhaps the best is The Art of UNIX Programming by Eric Raymond (Addison-Wesley, ISBN: 0-13-142901-9), which focuses on the philosophy behind UNIX and manages to mix in much about the command line.
O’Reilly’s UNIX CD Bookshelf (ISBN: 0-596-00392-7) contains seven highly respected books in one, although it retails for more than $120 as a result! That said, it is incomparable in its depth and breadth of coverage.
3.16.135.225