3 Linux and UNIX Overview Pretty Much Everything You Need to Know About Linux and UNIX to Follow the Rest of This Book

Introduction

My mistress’ eyes are nothing like the sun;
Coral is far more red than her lips’ red:
If snow be white, why then her breasts are dun;
If hairs be wires, black wires grow on her head.
I have seen roses damask’d, red and white,
But no such roses see I in her cheeks;
And in some perfumes is there more delight
Than in the breath that from my mistress reeks.
...
And yet, by heaven, I think my love as rare As any she belied with false compare.

—William Shakespeare, Sonnet 130

To understand how numerous attacks function, it’s very helpful to have a basic knowledge of the Linux and UNIX operating systems because they are so popular as target platforms and as operating systems from which to launch attacks. This chapter presents an overview of the Linux and UNIX operating systems, describing underlying concepts that are required to understand numerous attacks throughout the rest of this book.

UNIX is a beautiful but strange beast. Originally introduced as a research project at AT&T more than 35 years ago, the UNIX operating system is widely used throughout the world on servers and workstations. Much of the Internet was built using UNIX, and UNIX systems remain incredibly popular as Internet hosts. In recent years, open-source UNIX and UNIX-like environments (such as OpenBSD, Linux, and others) have helped to push UNIX to the desktop and even to palmtop devices.

UNIX is beautiful because it is so powerful. Millions of people have worked on developing UNIX over the years, optimizing routines and creating a huge number of useful tools. A variety of kinks that often plague new operating systems have been worked out in the decades-old UNIX. This operating system has clearly been around the block a couple of times. Because of this, many UNIX systems have great reliability, high levels of performance, and strong security features. Given UNIX’s origins as a research tool, its close relationship with the Internet, and critical role in the free software and open-source movements, system administrators can find a variety of tools freely available on the Internet and can ask questions of a large and relatively friendly community of UNIX system administrators and users through mailing lists and newsgroups.

Although it is beautiful, UNIX is also a strange beast, for two reasons in particular. First, there is no single operating system called UNIX. Instead, UNIX is a family of operating systems, with members of the family constantly being updated by many competing vendors, individuals, and even standards bodies with different visions and goals. Several popular variants of UNIX include the following:

• Solaris by Sun Microsystems
• MacOS X by Apple Computer
• HP-UX by Hewlett Packard
• IRIX by sgi (the new name for Silicon Graphics)
• AIX by IBM
• FreeBSD, a free, open-source version of the Berkeley Software Distribution (BSD) variant of UNIX
• OpenBSD, another free variant of BSD whose goal is to “Try to be the #1 most secure operating system”

This list just represents some of the UNIX variations available today. Although they might have the same genetic root in the first AT&T UNIX of decades ago, the members of this family were clearly raised by vastly different parents, some of whom nurtured their UNIX to be computing virtuosos, whereas others appear to have severely neglected their descendents. Of course, the UNIX variation that one person considers the absolute best and most elegant is often considered horrible and outdated by another person. Arguments about which is the best UNIX variation often turn into pseudo-religious flame wars.

File system organization, system calls, commands, and options within commands differ for different types of UNIX. There are two main lines in the UNIX family: the AT&T and BSD lines. Most UNIX systems resemble one of these family lines more closely than the other. For example, Solaris and HP-UX machines tend to look more like the AT&T family line, whereas FreeBSD and MacOS X operate more like the BSD line. Of course, just to make things more complex, some systems, like IRIX and AIX, have interesting mixtures of both bloodlines and many additional nuances.

Linux tends to lean more toward BSD, but has some AT&T quirkiness thrown in for good measure. Strictly speaking, Linux, the open-source project spearheaded by Linus Torvalds, is not a variation of UNIX (that’s why we didn’t include it in the list above). Linux was created without using any of the underlying UNIX code (although some lawsuits have alleged otherwise, officially Linux doesn’t contain real UNIX code). Instead, Linux is a UNIX-like environment, which borrowed heavily from the ideas and tools developed in UNIX. Even more strictly speaking, the term Linux itself just refers to the kernel at the heart of various operating system distributions. Various vendors and open-source aficionados have built operating systems around the Linux kernel, with each separate flavor referred to as a particular Linux distribution, or “distro” for short. Some of the most popular Linux distributions include Debian, Gentoo, Mandrake, Red Hat, Slackware, and SuSE.

This chapter, and the rest of the book, tries to deal with generic Linux and UNIX concepts, focusing on ideas that apply across all members of the Linux and UNIX family, or at least most of them. When discussing these numerous Linux and UNIX types, many people refer to them as different UNIX flavors, variants, varieties, or even un*x. This book refers to them as Linux and UNIX flavors, variants, or varieties, using the terms interchangeably.

A second reason that many people consider Linux and UNIX to be strange beasts is that they have traditionally been—how shall I put this delicately—not optimized for ease of use. Reflecting their early roots, many varieties of Linux and UNIX do not shield their users from the complexity of the underlying system. Their user interfaces were, and to some extent still are, often not designed for complete GUI-based administration and use. Many aficionados love this command-line orientation in an operating system, an attitude I share. However, a new user accustomed to a GUI environment is often overwhelmed by the available command-line options. Also, interfaces and some underlying concepts vary greatly among UNIX flavors. For example, a grand master of Solaris might be nearly helpless in a Linux environment. Still, once mastered, the beautiful power of Linux and UNIX shines through. Truth be told, Linux and most UNIXes are actually far simpler than Microsoft Windows. Windows just tries to cover up its monstrous complexity under the veneer of a GUI. However, once you learn the guts of Windows, you’ll see that it really is more complicated than Linux or UNIX, with so many more options and quirks. If you think that Windows is easier than Linux, you probably don’t know Linux that well. What’s more, if you think that, it’s quite possible that you don’t know Windows that well either.

Learning About Linux and UNIX

In this chapter, we cover Linux and UNIX briefly to gain a grounding to understand attacks described throughout the rest of the book. If you want to get deeper into the guts of Linux and UNIX, I strongly recommend the excellent Linux Administration Handbook by Evi Nemeth (Prentice Hall, 2002).

Another incredibly useful source of information about Linux and UNIX is the online system documentation known as the man pages, which is an abbreviation for manual pages, and has nothing to do with the masculine gender. Linux and UNIX systems with man pages installed include detailed information about the usage and function of most system commands and critical system concepts. Often times, the man pages for a given program or features are written by the author of the program and tell you exactly what you need to know about a function. To look up a man page for a given command, simply type the following at a command prompt:

$ man [system command]

Architecture

Linux and UNIX File System Structure

Linux and UNIX are very much organized around their file system structure. Showing UNIX’s late 1960s and early 1970s vintage, darn near everything is treated as a file: many devices, certain elements of processes, and, of course, files. Exploring the Linux and UNIX file system is like traveling through a city, with different directories acting like streets to lead you to the buildings, which are individual files. Although some particular flavors of Linux and UNIX might have subtle variations, a high-level map of the UNIX file system is shown in Figure 3.1.

Figure 3.1 A high-level roadmap of the UNIX file system.

The tip-top of the UNIX file system is known as the “root” directory, simply because it’s at the top and all other directories are under it. I know, the roots are usually at the bottom of a tree, but this one is inverted, with the root directory at the top. The root directory is conveniently named /, which is often pronounced “slash.” By changing the directory to / (using the “change directory” command like this: cd /), you will find yourself at the top, overlooking all directories on the system. Every file is referred to on the system relative to this slash directory. So, the file hack.txt located in the /home/fred directory would be identified as /home/fred/hack.txt (pronounced “slash-home-slash-fredslash-hack-dot-T-X-T”). At the next level down from the root directory, a number of other directories hold the rest of the information on the machine, including system configuration, system executables, and user data, as described in Table 3.1.

Table 3.1 Important Directories in the Linux and UNIX File System

Two other directory names are of paramount importance in UNIX: the names “.” and “..”. These names don’t refer to just one directory in the file system, however. They are links included inside every directory to refer to the current directory and the parent directory, respectively. For example, if you are working in the /etc directory, you can use the ls -a command to list the contents of the directory. The -a indicates that you want to see all of the contents of the directory, including files with names that start with a dot. If you run ls without the -a option, these so-called dot-files will be omitted from the output, because they contain a lot of configuration information that could clutter a user’s experience. When you run ls -a, you will see . and .. in the output. The . refers to the current directory itself, in this example /etc. You can refer to files in this directory as ./filename when running commands. Likewise, the directory .. refers to the directory just above the current directory in the file system hierarchy, the parent directory. So, if you are in the /etc directory, and you refer to .., you are referring to its parent, which is the / directory.

Now that we have a high-level view of the file system structure, let’s analyze how the underlying operating system is organized.

The Kernel and Processes

Linux and UNIX systems tend to have a very modular architecture, with a central core and various programs around the core. On a Linux and UNIX machine, the special program at the core is called, appropriately enough, the kernel. The kernel is the heart and brain of the system, controlling critical system functions, such as interactions with hardware and doling out resources for various user and administrator programs running on the machine. When a running program needs to access hardware components, such as disks or network interfaces, it calls on the kernel, which provides the required functions to access the hardware.

When a program runs on a Linux or UNIX system, the kernel starts a process to execute the program’s code. A process contains the running program’s executable code, the memory associated with the program, and various threads of execution that are moving their way through the code executing its instructions. User programs, administrative tools, and even services (like Web servers or mail servers) are processes on the machine. Think of a process like a bubble that contains all of the guts of a running program. The kernel inflates the bubbles (by creating processes), controls the flow of bubbles, and tries to keep them from popping one another. A single UNIX system often has hundreds or even thousands of active processes at any given time. However, one Central Processing Unit (CPU) on a machine can run only one process at any given instant. The kernel juggles the CPU among all of the active processes, scheduling each one so that the system’s processor can be shared among the processes. Additionally, the kernel carefully allocates and manages the memory used by processes. Each process has its own limited view of memory, and the kernel prevents one process from accessing the memory used by another process. With this memory protection capability, a renegade process trying to read or overwrite the memory of another process will be stopped by the kernel.

Figure 3.2 contains a high-level diagram showing the relationship between processes, the kernel, and the systems hardware.

Figure 3.2 High-level view of generic UNIX or Linux architecture.

Many processes on Linux and UNIX systems run in the background performing critical system functions, such as spooling pages to be sent to a printer, providing network services such as file sharing or Web access, or providing remote management capabilities. These background processes are known as daemons, which is pronounced “day-muns” or “dee-muns,” depending on whom you ask.

Daemons are commonly given names based on the function they perform, followed by a “d” to indicate that they are a daemon. For example, the SSH daemon (known as sshd) allows users and administrators to access the system securely across the network using a command line. Similarly, httpd is a daemon providing HTTP access to the system, or, in more common parlance, a Web server.

Automatically Starting Up Processes: Init, Inetd, Xinetd, and Cron

All processes running on a Linux or UNIX system, from the mightiest Web server to the lowliest character generator, have to be activated by the kernel or some other process to start running. During system boot, the kernel first gets loaded into memory. Then, the kernel itself activates a daemon called init, which is the parent of all other user-level processes running on the machine. Init’s job is to finish the bootstrapping process by executing start-up scripts to finalize the configuration of the machine and to start up a variety of system processes. The location of these start-up scripts varies on different Linux and UNIX flavors, but /etc/init.d or /etc/rc.d are common locations. Init runs these shell scripts, which include capabilities for starting system logging, scheduling tasks for the machine, and initiating network interfaces.

Init also starts a bunch of processes associated with network services. These network service daemons are activated, listen on a specific port for incoming traffic, and interact with the network traffic. Some of the most common network services daemons started by init include the following:

• Httpd. A Web server, handling HTTP or HTTPS requests
• Sshd. The SSH service, offering strongly encrypted and authenticated remote shell access
• Sendmail. A common UNIX implementation of an e-mail server
• NFS. The Network File System, originally created by Sun Microsystems, used to share files between UNIX systems

We’ll discuss each of these services in a bit more detail at the end of this chapter. When the init daemon starts up one of these network services, the process associated with the service listens to the network for incoming traffic. For example, most Web servers listen on TCP port 80, and e-mail servers listen on TCP port 25. These processes just sit there and wait for incoming traffic to service.

Some network services, like Web, mail, and file sharing, usually have a lot of incoming traffic, so they need to be constantly ready to handle the incoming onslaught. Other services, like telnet or FTP, are usually not as frequently accessed. Having a large number of different processes just sitting around waiting for infrequent traffic is inefficient, because each infrequently accessed service requires system resources, including memory and some CPU time. To improve performance, some Linux and UNIX network services are not started by init and don’t just sit and wait for traffic. Instead, another process, called the Internet Daemon, or inetd for short, does the waiting for them. On some Linux and UNIX variations, inetd has been replaced with xinetd, an extended version that offers better access control and logging.

Either inetd (pronounced “I-Net-D”) or xinetd (that would be “X-I-Net-D”) are activated by the init daemon during the boot process. Once activated, inetd consults its configuration file, located in the /etc directory and called, appropriately enough, inted.conf. This configuration file tells inetd to listen on the network for traffic for a specific set of services. The TCP and UDP port numbers for these services are defined in the file /etc/services, which just contains a service name, port number, and indication of whether a service is TCP or UDP. For systems that use xinetd, several configuration files are used, one for each service that is started by xinetd, such as the telnet or FTP servers. These configuration files are typically located in the /etc/xinetd.d directory.

When traffic arrives at the machine destined for a specific service identified in its configuration file or directory, inetd or xinetd activates the process associated with the service. The particular network service process then handles the traffic and stops running when it is finished. Inetd or xinetd then continues to wait for more traffic for that service and others. Numerous services are commonly activated using inetd or xinetd, including the following:

• Echo. A service that just echoes back the characters sent to it, sometimes used to troubleshoot network connectivity problems.
• Chargen. A service that generates a repeating list of letters, sometimes used to measure performance.
• Ftpd. The FTP daemon, used to move files between machines.
• Telnetd. A telnet server for remote command-line access offered on a clear-text (and thus quite unsecure) basis.
• Shell, login. These are the UNIX r-commands for remote shell (rsh) and remote login (rlogin), respectively, which allow a user to execute commands and log in remotely to the system, again in a very unsecure manner.
• TFTP. TFTP, a bare-bones file transfer mechanism.

To make inetd listen for a particular service, an entry in the /etc/inetd.conf is required for each service. A sample inetd.conf file contains the following information (note that the “#” character indicates that a line is a comment and will not be processed by inetd):

The various fields of the inetd.conf file describe the particular characteristics of the service to be launched by inetd, and include, from left to right:

• Service name. This field refers to a specific service, such as telnet or FTP, which is defined in the /etc/services file. The /etc/services file is just a simple mapping of service names to TCP or UDP port numbers.
• Socket type. This field describes the type of connection used by the service, and can be set to stream, dgram (for datagram services), raw, rdm (for reliably delivered message), or seqpacket (for sequenced packet sockets). stream and dgram are by far the most commonly used values for TCP and UDP services, respectively.
• Protocol. The particular network protocol type is described here, usually tcp or udp. This field could also be set to rpc/tcp or rpc/udp to indicate an RPC service.
• Wait status. This field indicates whether a single server process can handle multiple requests at once. If so, this field is set to wait, preventing inetd from creating a bunch of processes to handle individual requests for the service. Otherwise, the field is set to nowait, so inetd will create one process to handle each incoming request.
• User name. This element gives the Login Name that the network service should run as. The network service will run with all of the permissions of this user.
• Server program. This field indicates which program to run to activate the network service.
• Server program arguments. This file field lists the arguments and configuration flags that should be passed to the network service when it starts to run.

To see how an attacker targets inetd.conf to create attack relays, please refer to the Chapter 8 section titled “Relaying Traffic with Netcat.”

The relationship between init and various daemons is shown in Figure 3.3. To summarize, there are two basic types of network services on a Linux or UNIX machine: services that are started by init and constantly wait themselves for traffic from the network, and services that use inetd or xinetd to listen for traffic and are activated only when traffic arrives for the service. The chkconfig command included in some Linux distributions can be used to display a list of all services configured to start up at system boot and by xinetd, by simply typing (as root):

Figure 3.3 The relationship between init, xinetd, and various network services.

# chkconfig --list

We look at the chkconfig command in more detail in Chapter 6, Phase 2: Scanning.

Beyond init, inetd, and xinetd, another way to automatically start processes is through the cron daemon. This daemon is used to schedule the running of specific system commands and programs at predetermined times. Administrators frequently use cron to schedule regular automatic processes to ease the job of system administration. If you want to run a program that scans the system for viruses every night at midnight or backs up the system at 3 AM, you will likely use cron to schedule the job. It reads one or more configuration files, known as crontabs, to determine what to run and when to run it. These crontab files are stored in different locations on various flavors of Linux and UNIX, but common locations include /usr/lib/crontab and /etc/crontab.

Just as system administrators use cron to get their work done, attackers also employ cron to accomplish their job of exploiting systems. Rather than manipulating inetd or xinetd on the victim machine to set up remote access, an attacker with access to a victim machine could edit the crontab files to run various commands on the victim. Such commands could include a denial-of-service attack program shutting down critical system services at a specified time, a backdoor listener granting remote access to the machine, or any other kind of timed attack against the system.

For more information about a variety of denial-of-service attacks, please refer to Chapter 9, Phase 3: Denial-of-Service Attacks.

For more information about placing backdoor listeners on a victim machine, please refer to the Chapter 10 section titled “Backdoors.”

Manually Starting Processes

Init, inetd (or xinetd), and cron automatically start processes running on a machine. Of course, users and administrators can manually start processes as well. Whenever you run a program on a Linux or UNIX machine by typing its name at the command line, a process is started to execute the program. When a user runs a program, the resulting process typically executes with the permissions of the user that activated the program.

When a user types a program name at a command prompt, the system looks for the program in a variety of directories that can be custom-tailored for that specific user. The directories searched for the program make up the search path for that user, or simply the path. The user’s search path is really just a variable that contains all of the directories that are searched by default, with each directory in the path separated by a colon. To see the setting of your search path, type the following command at a command prompt:

$ echo $PATH

You will get a response similar to this:

/usr/local/bin:/bin:/usr/bin:/usr/X11R6/bin

This response indicates that when I type a particular program’s name, the system will attempt to find the program first in the /usr/local/bin directory, then in the /bin directory, then /usr/bin/, and finally the /usr/X11R6/bin directory. If it cannot find a program with the name I typed in those directories, the system responds with a “command not found” message.

It is very dangerous to have the current working directory, ., in your search path. To understand why, consider what happens when you type a normal command, such as ls to get a listing of the contents of the current directory, but you have . in your search path. If the . in your path comes before the directory where the real ls program is located, you will unwittingly execute a program named ls in your current directory. This program could be anything that happens to be named ls. Attackers love to see . in someone’s search path. If it’s there, an attacker can put an evil program with a name of a commonly used command (like ls) in one of your most often used directories to trick you into executing it. The evil program that the attacker tricks you into executing might be a backdoor, password stealer, denial-of-service attack, and so on. By default, all major Linux and UNIX distributions leave . out of your path, and you should keep it that way!

Interacting with Processes

The kernel assigns each running process on a machine a unique process ID (called a PID, and often pronounced “P-I-D”), which is a number used to reference the process. Users can run the ps command to display a list of running processes. The ps command can also be used to show the PID, program names, CPU utilization, and other aspects of each running program. To show the details of all running processes on a system with BSD characteristics, use the -aux flags with the ps command. For a UNIX machine with AT&T family characteristics, the -edf flags give a detailed display of all running processes.

Here is an example of the output from the ps command run on a typical Linux installation (note that I’ve edited out several processes and bolded others from this list to make it easier to read). In the following list, you can clearly see the init, crond, and xinetd processes running on the system. Additionally, the user’s command shell (bash) is a process, as is the ps command itself that is run to generate the list of processes.

Beyond ps, another really handy command for looking at what processes are up to is lsof, a command included in many Linux and UNIX variants, and available as a separate download for others. This command merely gives a “list of open files” for each running process on the machine. But, remember, on a Linux or UNIX system, pretty much everything is treated as a file, including files (of course), terminals, and even network ports (the TCP and UDP ports we discussed in Chapter 2: Networking Overview). You can run lsof by itself to get an enormous amount of information about every file that every process on your system is accessing. You can then take the output of lsof and feed it as input using a pipe to the grep command to find specific patterns, such as the string “bash” by running lsof | grep bash. That way, you’ll see all of the bash command shells on your machine. Then, to zoom in on a specific process, you can run lsof -p [pid] to see all of the files that a specific process is currently accessing. Finally, the command lsof -i shows all TCP and UDP port usage on a machine, as well as the processes using those ports. We look at lsof in more detail in Chapter 6.

One way to interact with processes is to send them a signal. A signal is a special message that interrupts a process telling it to do something. One of the most common signals is the TERM signal (short for terminate), which instructs the process and the kernel to stop the given process from running. Another frequently used signal is the hangup signal (HUP), which causes many processes (particularly inetd or xinetd) to reread their configuration files. A user can run the kill command to send a signal to a specific process, by referring to the PID. Similarly, the killall command is used to send a signal to a process by referring to its name on Linux systems. Be careful with the killall command on Solaris, because it will do what its name implies: kill everything! On Linux, for example, suppose an administrator or attacker alters the configuration of xinetd by making a change to one of the files in the directory /etc/xinetd.d. To make the changes active on the system, xinetd must be forced to reread its configuration. To cause the xinetd process from the process list shown previously to reread its configuration files, an administrator or attacker could use the kill command to refer to its PID:

# kill -HUP 462

Or, alternatively, on Linux, the administrator could use the killall command to refer to the process name:

# killall -HUP xinetd

Now that we have an understanding of processes, let’s turn our attention to other fundamental UNIX concepts, accounts, and groups.

Accounts and Groups

To log in to a Linux or UNIX machine, each user must have an account on the system. Furthermore, every active process runs with the permissions of a given account. Without these accounts, no one can log in and no processes can run. Clearly, in Linux and UNIX, to get anything done, an account is required. Let’s analyze how accounts are configured.

The /etc/passwd File

Accounts are created and managed using the /etc/passwd file, which contains one line for each account on the machine. An example /etc/passwd file might contain the following information:

Each line in the /etc/passwd file contains a description of one account, with parameters separated by a colon (:). The parameters included in /etc/passwd for each account are, from left to right:

• Login name. This field contains the name of the account. A user logs into the machine using this name at the login prompt.
• Encrypted/hashed password. This field contains a copy of the user’s password, cryptographically altered using a one-way function so that an attacker cannot read it to determine users’ passwords. Various cryptographic algorithms are used on various Linux and UNIX flavors, including hash algorithms and encryption ciphers. When the user logs into the machine, the system prompts the user for the password, applies the one-way cryptographic function to the user-supplied password, and compares the result with the value stored in /etc/passwd. If the encrypted or hashed password provided by the user matches the encrypted or hashed password in /etc/passwd, the user is allowed to log in. Otherwise, access is denied.
• UID number. Each account is assigned an integer called the user ID number. All processes and the kernel actually rely on this number and not the login name to determine the permissions associated with the account.
• Default GID number. For the purposes of assigning permissions to access files, users can be aggregated together in groups. This field stores the default group number to which this account belongs.
• GECOS information. This field is filled with free-form information not directly referenced by the system. It is often populated with general information about the user, such as full name and sometimes telephone number.
• Home directory. This value indicates the directory the user is placed in after logging into a system, the starting directory. It is often set to a directory in the file system where the user’s own files are stored, often in /home.
• Login shell. This field is set to the shell program that will be executed after the user logs into the system. This field is often set to one of the command-line shells for the system, such as the bourne shell (sh), the bourne-again shell (bash), C shell (csh), or Korn shell (ksh). It could also be set to another program to be executed when the user logs in.

The /etc/passwd file is world-readable, so any user or process on the system can access it. Because some attackers read the password file and attempt to recover the encrypted or hashed passwords through password cracking techniques, many modern Linux and UNIX systems do not include the encrypted or hashed passwords in the world-readable /etc/passwd file and instead store passwords in a socalled shadow password file. Ironically, on systems with a shadow password file, the /etc/passwd file doesn’t contain any passwords. Instead, /etc/passwd uses the same format and holds all of the other information defining accounts, except the password hash is removed from the file. A * or an x is placed in the location where the password would be located. The encrypted or hashed passwords themselves, on such Linux and UNIX systems, are relocated to the shadow password file called /etc/shadow or /etc/secure. Access to the shadow passwords is carefully guarded, as only users with super-user privileges can access the encrypted or hashed passwords by reading the shadow file.

To see how an attacker tries to determine passwords through guessing and cracking to gain unauthorized access to a system, refer to the Chapter 7 section titled “Password Attacks.”

The /etc/group File

When administering a system, handling the permissions of each individual user account can be a lot of work. To help simplify the process, Linux and UNIX include capabilities for grouping users and assigning permissions to the resulting groups. All groups are defined in the /etc/group file, which has one line for each group defined on the machine. A common /etc/group file might look like this:

The format of the /etc/group file includes the following fields, each separated by colons:

• Group name. This field stores the name of the group.
• Encrypted or hashed group password. This field is never used, and is frequently just set to an x or a *.
• GID number. This value is used by the system when making decisions about which group should be able to access which files.
• Group members. The login name of each user in the group is included in this comma-separated list. In the example /etc/group file listed earlier, the root and bin accounts are all in the daemon group, which has a GID of 2. Similarly, the owners of the alice, fred, and susan accounts are all in the finance group, with a GID of 25.

Root: It’s a Bird ... It’s a Plane ... No, It’s Super-User!

The single most important and powerful account on Linux and UNIX systems is the root account, usually named root. Root has the maximum privileges on the machine; it can read, write, or alter any file or setting on the system. With these great privileges, root is sometimes referred to as the super-user or even “god” account. The UID number of a root account is zero. When the system checks to see if a given action requires super-user privileges to execute, it consults the UID of the user or process requesting the action. Therefore, the super-user account could be named anything (although root is most common) as long as the UID is zero. Multiple UID 0 accounts are possible on a single system, with each having super-user access at the same time. System administrators use the root account to manage the system. Attackers love to gain root access on a machine, because it allows them complete control over the machine.

Linux and UNIX Permissions

Each file in the Linux or UNIX file system has a set of permissions describing who can access the file and how they can access it. Every file has an owner (a single account associated with the file) and an owner group (a single group associated with the file). The owner of the file (along with root) can set and alter the permissions of the file.

Linux and UNIX file permissions are broken down into three areas: permissions associated with the owner of the file, permissions assigned to the owner group, and permissions for everyone (i.e., all users and processes with accounts on the machine). For each of these three areas, at least three kinds of access are allowed: read, write, and execute. With three areas (owner, group owner, and everyone) and three different levels of access (read, write, and execute), there are nine different standard permission settings. Using the ls command, with the -l flag to look at the long form of the output, we can see the permissions assigned to the files in a given directory, as in the following example:

Note that each item in the listing begins with a pattern of ten characters. If the first character is a d, it indicates that the associated listing is a directory. Otherwise, it is a file. The next nine characters indicate the permissions for each directory, using the format shown in Figure 3.4.

Figure 3.4 Linux and UNIX file permissions.

When an r, w, or x permission is allowed, the appropriate letter is displayed in the output of the ls -l command. When the given permission is not allowed, a - is shown in the ls -l output.

These permissions for each file can be altered using the chmod command, pronounced “ch-mod.” This command’s name is a reference to changing the “modes” of a file, another way of referring to access permissions. To change the permissions of a file, a user can convert the desired permissions to octal format and enter the result into the chmod command. Figure 3.5 shows how the desired permissions are converted to octal representations. First, the desired permissions are listed as a sequence of nine bits. A zero bit means that the capability is absent, and a one bit means the capability is present. Then, each bundle of three bits is converted into octal format (see Table 3.2).

Figure 3.5 Permission assignments.

Table 3.2 Octal Equivalents for File Permissions

For example, suppose we want a file named foo to have full control (read, write, and execute) capabilities for its owner account, we want it to be readable by the owner group, and we want everyone to be able to read and execute it. The desired permission set would be rwxr--r-x, or converted to binary, 111 100 101. The resulting octal representation would be 745. We set these permissions using this command:

# chmod 745 foo

As perverse as it might sound to UNIX neophytes, with enough use of these octal formats, your brain eventually maps the rwx permissions to their octal representations and back automatically. For the octally challenged, the chmod command on most UNIX flavors also allows users to type in the individual r, w, and x permissions for the owner account, owner group, and everyone by hand, a more painstaking process. Most people use the octal representation.

SetUID Programs

Sometimes, users or processes have a legitimate reason for accessing a file for which they don’t have assigned permissions. Consider what must happen for users to change their own passwords. The user has to edit his or her account entry in the /etc/passwd or /etc/shadow file. However, the /etc/passwd or /etc/shadow files can only be altered with super-user-level permissions. How can a lowly user change a password without having to pester the system administrators every time to use their rootly powers to modify the password file on behalf of the user?

The answer lies with another Linux and UNIX capability called SetUID (for Set User ID). With this capability, a particular program can be configured always to execute with the permissions of its owner, and not the permissions of the user that launched the program. Remember, usually, when a user starts a process, the process runs with the user’s permissions. SetUID programs alter this, allowing a user to run a process that has the permissions of the program’s owner, and not the user executing the program.

So, in our password-changing example, the user can run a special SetUID program called passwd to change a password. The passwd program is configured to run SetUID root. That is, regardless of who executes the passwd program, it runs with root permissions. The passwd program asks the user for the new password and overwrites the /etc/passwd or /etc/shadow files with the new encrypted or hashed password. The passwd program then finishes running, and the normal user has finished the encounter with root privileges.

SetUID capabilities give common users temporary and controlled access to increased permissions so they can accomplish specific tasks on the system. Set-UID programs are identified with a special additional bit in their permissions settings. This bit is actually located before the nine standard permissions (rwxrwxrwx). In fact, there are three additional bits that can be used in addition to the nine standard permissions. These bits are the SetUID bit, the SetGID bit (so a program can run with the permissions of its owner group rather than the group of the user that launches it), and the so-called sticky bit, which forces programs to stay in memory and limits deletion of directories. Just like the nine permission bits, the SetUID, SetGID, and sticky bits are converted to an octal number to be used in a chmod command. In the octal representation, SetUID comes first, followed by SetGID, followed by the sticky bit.

Therefore, to change the file from our earlier example, foo, to run SetUID, the owner of the file (or root) could type:

# chmod 4745 foo

The leading “4” is the octal equivalent of the binary “100,” meaning that the SetUID bit is set, whereas the SetGID bit and sticky bit are not. The remaining permissions (745) are identical to what we had set them to in our earlier example.

When the ls command is used to display permissions, it does indicate which files are SetUID by overwriting the x for the file’s owner with an s character, as shown in the following example:

If you think this idea of allowing lowly users to run programs with great permissions is a little bit scary, you’re absolutely right. Any program that is SetUID, particularly those that are SetUID root, must be carefully constructed to make sure that a user cannot exploit the program. If attackers have an account on a system and can run SetUID programs, they can attempt to break out of the SetUID program to gain increased privileges. The attackers might try to provide bogus input to the SetUID program or even crash it in an attempt to gain elevated privileges. Because of this possibility, SetUID programs must be carefully written to minimize the access given through the program to the user. Furthermore, system administrators should maintain an inventory of all SetUID programs on a machine. Newly added or modified SetUID root programs could be an indication that an attacker is present on the machine, and has set up a SetUID root program as a quick way to jump to root. To find all SetUID programs on a UNIX machine, you can run the following command as a root-level user:

To understand a common technique used by attackers to exploit vulnerable applications that is particularly effective in breaking SetUID programs, please refer to the Chapter 7 section titled “Buffer Overflow Exploits.”

Linux and UNIX Trust Relationships

Now that we’ve seen how accounts and permissions work on a single system, let’s analyze how access can be extended between Linux or UNIX machines. Linux or UNIX systems can be configured to trust each other, an operation that can make the systems simpler to administer, but potentially impacting security. When one system trusts another, it allows the trusted system to authenticate users on its behalf. As shown in Figure 3.6, machine Bob trusts machine Alice. When a user logs into Alice, that user can send commands to be executed on Bob, and Bob will not require the user to reauthenticate. The user will not see a password prompt on Bob, because Bob trusts the fact that Alice has already authenticated the user. System Bob thinks, “Well, if my friend Alice, whom I trust, has already authenticated this user, that’s good enough for me!”

Figure 3.6 Bob trusts Alice.

This trust can be implemented in Linux and UNIX systems using the systemwide /etc/hosts.equiv file or individual users’ .rhosts files, along with a series of UNIX tools known collectively as r-commands. The /etc/hosts.equiv file contains a list of machine names or IP addresses that the system will trust (i.e., allow unauthenticated access of users from the given machine). Similarly, users can create a file called .rhosts in their home directories setting up trust between machines. The r-commands include rlogin (a remote interactive command shell), rsh (a remote shell to execute one command), and rcp (a remote copy command), among others. Each of these commands allows for remote interaction with another machine. If the remote machine trusts the system where these commands are executed, no password is required for the remote access. The r-commands are incredibly weak from a security perspective, as they base their actions on the IP address of the trusted machine and carry all information in clear text.

To see how an attacker undermines the UNIX r-commands, please refer to the Chapter 8 section titled “IP Address Spoofing Flavor 2: Predicting TCP Sequence Numbers to Attack UNIX R-Commands.”

Because of their weaknesses, the r-commands should be replaced with more secure tools for extending system trust, like the SSH tool, which provides for strong, cryptographic authentication and confidentiality, as discussed in Chapter 2. Wherever possible, move away from the rlogin, rsh, and rcp commands, and use the SSH tool. Otherwise, you’re asking for problems. Sometimes, we see an old legacy system that doesn’t support SSH, and is trapped in the older and weaker rsh, rlogin, or rcp world. If you are faced with such a dilemma, you might consider putting a modern system in front of the legacy box, and tunneling the weak r-commands over a more secure tunnel, such as an IPSec encrypting VPN, or even an SSH tunnel. That way, the security of the encrypted tunnel will help shore up the weaknesses of rlogin, rsh, and rcp.

Logs and Auditing

To detect attacks on a Linux or UNIX system, it is important to understand how various logging features work. In Linux and UNIX systems, event logs are created by the syslog daemon (known as syslogd), a process that sits in the background and receives log information from various system and user processes, as well as the kernel. The syslogd configuration is typically contained in the file /etc/syslog.conf, which specifies where the log files are placed on the system. Although particular Linux and UNIX flavors might store logs in different locations, the directory /var/log is a popular location for the logs. Although the particular log files vary for different variants of Linux and UNIX, some common log files of interest include the following:

• Secure (such as /var/log/secure). This file contains information about successful and failed logins, including the user name and originating system used for login. Login records for applications such as telnet, rlogin, rsh, ftp, and so on are stored in this file. Different flavors of Linux and UNIX might or might not have this file, or might store the information under a different name.
• Messages (such as /var/log/messages). This file contains general messages from a variety of system components, including the kernel, specific modules, and daemons. It acts as sort of a catch-all for system logs.
• Individual application logs (such as /var/log/httpd/, /var/log/cron, and so on). Whereas some applications send their logs to a general log file (such as /var/log/messages), others have specific log files. A common example is Web servers, which can be configured to log HTTP requests and other events to their own log files.

The vast majority of log files in Linux and UNIX are written in standard ASCII, and require root privileges for modification.

In addition to the system log files, Linux and UNIX also store information about user access in various accounting files, which are used by system administrators and (sometimes) users to detect anomalous activity. Furthermore, forensics investigators can use these accounting files during investigations. To foil detection by system administrators and users, as well as undermine forensics investigations, the following accounting files are of particular interest to attackers desiring to cover their tracks:

• utmp. This file stores information about who is currently logged into a system. When a user or administrator types the who command, the operating system retrieves the contents of the utmp file to display who is logged in. A complete list of all users logged into the system is displayed, which is bad news for an attacker wanting to hide. Depending on the flavor of UNIX, this file can be stored in /var/run, /var/adm, or other locations.
• wtmp. This file records all logins and logouts to and from the system. Depending on the flavor of UNIX, this file can be stored in /var/log, /var/adm, or other locations. The command last displays a list of all users that have logged in to the system, using the contents of wtmp.
• lastlog. The lastlog file contains information about the time and location of each user’s last login to the system. On many Linux and UNIX systems, when a user logs in (by telnetting, using SSH, or accessing the system from the console), the system consults the lastlog file to display a message saying something like, “Last login for user Joe was at 3:35 AM from machine ftp.hacktheworld.com.” The purpose of these messages is to aid users in detecting misuse of their accounts: “What!!?!?! I never logged in at 3:35 AM from a machine called ftp.hacktheworld.com!” Unfortunately, the vast majority of users don’t pay very close attention to messages scrolling by on the screen while logging in, and would never notice or report such a message. On many Linux systems, the lastlog file is located in /var/log/lastlog. On some Linux variants, administrators can analyze the lastlog file using the lastlog command to see when each user last logged in and where they came from.

To see how an attacker manipulates these audit logs, refer to the Chapter 11 section titled “Attacking System Logs and Accounting Files in Linux and UNIX.”

Common Linux and UNIX Network Services

Most Linux and UNIX systems include a standard complement of network services. Because vendors are often more interested in ease of use rather than security, the default installation of many Linux and UNIX systems leaves many of these services active, waiting for user (and attacker) connections. To properly secure a system, you should deactivate or remove all services that are not explicitly required on the machine. To determine which services you might or might not require on a Linux or UNIX machine, let’s analyze some common services in more detail.

Although there are thousands of possible services that can be run on a Linux or UNIX machine, the purpose of this section is to describe a handful of the most commonly used and exploited services. It is important to note that many of the services listed in this section originally came into prominence on UNIX systems, but are now widely supported on a variety of machines. In particular, Linux and Windows now support most of these services that were once associated mostly with UNIX.

Telnet: Command-Line Remote Access

Telnet provides a command-line interface to a system remotely across the network. Users type in their user ID and password into a telnet client, which carries the information to the telnet server. On most Linux and UNIX systems, the telnet server (known as telnetd) is invoked by inetd or xinetd. With standard telnet, all information is carried without encryption (in clear text), and can be easily captured off the network by an attacker. Furthermore, telnet sessions can be easily taken over by an attacker in a session-hijacking attack.

To see how an attacker uses a sniffer to gather information from a network, please refer to the Chapter 8 section titled “Sniffing.”

To see how an attacker hijacks connections, please refer to the Chapter 8 section titled “Session Hijacking.”

FTP: The File Transfer Protocol

FTP is used to move files between systems. Like telnet, FTP servers are typically started by inetd or xinetd, and all data is transmitted in clear text. Because FTP sessions are not encrypted, they can be easily captured by an attacker and even hijacked.

To see how an attacker bounces a scan off of an FTP server, please refer to the Chapter 6 section titled “Obscuring the Source: FTP Bounce Scans.”

A Better Way: Secure Shell (SSH)

The very sniffable and hijackable telnet and FTP services can be bad news from a security perspective. A better approach for remote command shell access and file transfer involves using the SSH tool. This program consists of a client (ssh) and a server (sshd) that allow for strongly authenticated and encrypted communication. On Linux and UNIX, the sshd program is typically started by init, and not inetd or xinetd. The SSH authentication can take place with a password, transmitted across the network in an encrypted form. Even stronger SSH authentication is available using public key encryption, where the sshd and each user has a public–private key pair. These keys are used to identify the users to the sshd and vice versa, as well as to exchange crypto keys for encrypting all data sent across the session. In this configuration, no password is ever transmitted across the network, even in encrypted form.

SSH clients and servers can communicate using two flavors of the SSH protocol: versions 1 and 2. The latter of these two is far more secure. As we’ll discuss in Chapter 8, Phase 3: Gaining Access Using Network Attacks, you should consider configuring all of your sshd installations to accept only SSH protocol version 2 on those systems that support it (most do today).

To see how attackers can trick users into making unsecured SSH protocol version 1 connections, refer to the Chapter 8 section titled “Sniffing HTTPS and SSH.”

In addition to offering command shell access and transferring files (via the scp program commonly bundled with SSH), SSH can also carry any TCP-based service in an encrypted fashion across the network using a technique called SSH port-forwarding. Using this technique, security can be added to many applications, riding across a rock-solid encrypted SSH tunnel.

Web Servers: HTTP

Web servers are used to send information to Web browsers using HTTP. The most popular Web server on Linux and UNIX today is the free Apache Web server (available at www.apache.org). Web servers are typically started by init. Because they are often publicly accessible across the Internet, Web servers are frequent targets of attackers.

For a description of a full-featured CGI scanner useful in locating vulnerable Web servers, please refer to the Chapter 6 section titled “Nikto: A CGI Scanner That’s Good at IDS Evasion.”

To understand a variety of application-level attacks against Web-based services, please refer to the Chapter 7 section titled “Web Application Attacks.”

Electronic Mail

A variety of mail servers are available for Linux and UNIX systems. One of the most popular mail servers is sendmail, a program available on both a commercial basis (from www.sendmail.com) and on a free basis (at www.sendmail.org). Years ago, sendmail had a variety of security problems, many of which allow an attacker to gain root-level privileges on a vulnerable machine. If you run sendmail (or any other mail server, for that matter) on your systems, make sure to apply security patches as your vendor releases them.

r-Commands

As described earlier in this chapter, r-commands such as rlogin, rsh, and rcp are sometimes used to interact remotely with Linux and UNIX systems. Each of these services is started by inetd or xinetd, and can offer an attacker an avenue for undermining Linux and UNIX trust relationships.

To see how an attacker undermines the r-commands, please refer to the Chapter 8 section titled “IP Address Spoofing Flavor 2: Predicting TCP Sequence Numbers to Attack UNIX R-Commands.”

Domain Name Services

Clients use DNS servers to resolve domain names into IP addresses, among other capabilities. By far, the most popular DNS server on Linux and UNIX systems is the Berkeley Internet Name Domain (BIND) server, often called named. On Linux and UNIX, DNS servers are usually started with init, and run in the background listening for requests. DNS is an incredibly important service. In their excellent book, DNS and BIND, Paul Albitz and Cricket Liu (O’Reilly, 2004) say, “Almost all business that gets done over the Internet wouldn’t get done without DNS.”

I couldn’t agree more. Think about it: If an attacker can take down your DNS servers or, worse yet, remap your domain name to another IP address, he or she could seriously undermine access of your systems on your internal network or across the Internet.

To see how an attacker gathers information from a DNS server to use in mounting an attack, please refer to the Chapter 5 section titled “The Domain Name System.”

To see how an attacker can send spurious DNS responses to redirect traffic on a network, please refer to the Chapter 8 section, “Sniffing and Spoofing DNS.”

The Network File System (NFS)

Linux and UNIX machines can share components of their file systems using the Network File System (NFS). Originally created by Sun Microsystems in the mid-1980s, NFS allows users to access files transparently across the network, making the remote directories and files appear to the user as though they were local. By simply changing directories, a user can access files transparently across the network using NFS. On the machine where the files to be shared are located, the NFS server exports various components of the file system (such as directories, partitions, or even single files). Other machines can mount these exports at specific points in their file systems. For example, one machine may export the directory /home/export so other machines can access the files in that directory. Another system can mount the exported /home/export directory onto its file system at the /mnt/files directory. A user on the second machine simply has to change directories to /mnt/files to access the remote files, without having to go through the explicit transfer of files that FTP would require.

On most Linux and UNIX systems, mountd is responsible for handling mount requests. Once an exported directory is mounted, the nfsd daemon is the process that works with the kernel to ship the appropriate files across the network to NFS clients.

Regardless of the flavor of UNIX or Linux, exporting files via NFS can be dangerous. If you share files too liberally, an attacker might be able to access data in an unauthorized fashion. Attackers frequently scan networks looking for worldaccessible NFS exports to see if any sensitive data can be read or altered. To prevent this type of attack, you should share only those portions of your file system with an explicit business need for sharing, export files only to hosts requiring access, and carefully assign permissions to the shared files. NFS sharing across the Internet is especially dangerous, and should be avoided. I much prefer to see someone use the secure copy capabilities of SSH or an IPSec-based VPN, as described in Chapter 2. Although not as transparent as NFS, such mechanisms are far more secure, having strong authentication and encryption capabilities.

X Window System

The X Window System, known as X11 or even simply as X, provides the underlying GUI on most Linux and UNIX systems. An X server controls the screen, keyboard, and mouse, offering them up to various programs that want to display images or gather input from users. One of the most commonly used X programs is the X terminal, which implements a command-line interface to run a command shell in a window on an X display. Attackers can abuse X in a variety of ways. To prevent such attacks, you should lock down your X displays using the xhost command or X magic cookies, which limit who can connect to your display and see the data on your screen. Going further, you can tunnel all X Window traffic across an SSH session, giving you encryption and stronger authentication. Also, if your machine does not require a GUI (such as a server with a dumb terminal as a monitor or a box managed entirely via a Web-based administration tool), delete the X Window software so an attacker cannot attack the system using X.

Conclusion

UNIX systems have been incredibly popular for more than three decades. Linux is a UNIX-like operating system that has gained incredible popularity in the last ten years. The power and integrated networking capabilities of Linux and UNIX have certainly helped fuel the growth of the Internet. With this great power and widespread use on the Internet, Linux and UNIX systems are common targets of attackers. Furthermore, Linux and UNIX have become an extremely popular platform from which to run attacks. An attacker can build a powerful Linux or UNIX workstation on an inexpensive PC and use it to attack all varieties of machines, including Windows, Linux, UNIX, and various other platforms. With its power and capabilities, Linux and UNIX are the platform of choice for many attackers.

Summary

It is important to understand Linux and UNIX because they are so widely used on servers and workstations today. Because of its flexibility, relatively high performance, and power, many attackers also use it as a base from which to launch attacks. Many flavors of Linux and UNIX are available today, each with different features, programs, and controls.

Linux and UNIX are organized around their file system, with most of the operating system designed around making as many entities on the system look like directories and files as possible. The top of the UNIX file system is the / directory, referred to as the “slash” directory. Under this directory, a variety of other directories include all system information. Important directories include /etc (which stores system configuration) as well as /bin and /sbin, which store important system executables.

The kernel is the heart of Linux and UNIX operating systems, controlling all interaction with hardware and between running programs. When a program is executed, a process is created to contain its code, working memory, and various threads of execution. Processes can be started in a variety of ways. The init daemon starts processes during system boot-up. The inetd or xinetd program listens for incoming network traffic and starts processes to handle it. Cron starts processes at prespecified times. Manual user interaction also can start processes. The ps command provides a list of running processes on a system, and lsof provides a wealth of information with its list of all files opened by all processes. Users and administrators can interact with processes by sending them signals using the kill and killall commands. The killall command must be used with care, because on Linux, it’ll kill processes with certain names, but on Solaris, killall shuts down the system entirely.

Accounts are defined in the /etc/passwd file. On some Linux and UNIX systems, the passwords are stored in /etc/shadow, a file that is readable only by accounts with super-user privileges. Groups are defined in /etc/groups. The root account has a UID of 0 and has full privileges on a Linux or UNIX system. Other accounts can have a UID of 0, and they too will have the same super-user privileges as the root account.

Read, write, and execute permissions are assigned to each file in rwxrwxrwx format, where the first three characters refer to the file’s owner, the second set of three characters refers to the owner group, and the third set of three characters applies to everyone on the machine with an account. The permissions can be altered using the chmod command, with the desired permissions provided in octal format.

The ls command shows the contents of a directory, with the -a option showing all files (including those whose names start with a dot), whereas the -l option shows the long form of the output, including the permissions associated with each file or directory.

SetUID capabilities allow a user to run a program with the permissions of the program’s owner. Although essential for running a Linux or UNIX system, Set-UID programs must be carefully guarded, as attackers frequently add or alter them. Linux and UNIX trust relationships allow a user on one machine to access a trusting system without providing a password. The UNIX r-commands, including rsh, rlogin, and rcp are often used with trust relationships and have major security weaknesses. These very weak services should be avoided, with users and administrators relying on the far stronger SSH tool for strongly authenticated, encrypted remote shell access.

Event logs are created by the syslog daemon, which stores most logs in standard ASCII format. Accounting entries, such as who is currently logged in and when each user last logged in, are stored in the utmp, wtmp, and lastlog files.

Most Linux and UNIX systems are prepackaged with a large number of network services active. Each of these services could have security risks. Therefore, all network services should be deactivated, except those that have an explicit business need on a machine.