Third Defensive Layer: Set the Rules for the Playground

This layer and the next could just as easily be considered layers 3a and 3b. In this layer, we attempt to prevent an attacker from exploiting holes in applications or the operating system to launch applications. In the next layer, we look into preventing an attacker who has successfully escalated his privileges from accessing stored data.

Patching

Applying the latest security patches will prevent an attacker from using known exploits to compromise your system. Running workstations and servers without patching them or letting patches sit uninstalled for a prolonged period is a recipe for disaster. It is like leaving a ladder outside the house that a potential burglar can use to break in. Sometimes exploits are discovered, documented, and the vendor is informed so that it can prepare a patch as a defense for its customers. In many cases, exploits are documented and published on the Internet before informing the vendor and sadly long before a fix is available. In reference to “The Playground” in the title for this defensive layer, patching and patch management dictates what “toys” can be played with and which ones cannot.

A prominent example occurred in 2001. The Nimda virus and Code Red and Code Red II worms exploited well-known and long-solved vulnerabilities in IIS. The initial news coverage focused on the speed and virulence of the infections. Nimda was reported to have become the most widespread virus in the space of 22 minutes. The reason that they were so successful was that many server administrators had failed to patch their IIS servers with a set of security patches that had been available for over one year. Once the reason was discovered, news coverage turned to administrators' procrastination and negligence, questioning the inherent security of IIS, and the need for more rigorous patch management.^[3]

From a security perspective, the key to effective patch management is reducing the amount of time between the release of the patch and the installation of the patch on affected servers and workstations. While speed is important, it has to be balanced with the need to test the patch (or collection of patches) to ensure that it does not destabilize the platform. The risk (probability and impact) of being exposed to attack needs to be weighed against the risk of interrupting business operations if a patch is rushed into the production environment without the proper care exercised with its release.

Data Execution Prevention

As mentioned in the section “Second Defensive Layer: Privileges Must Be Earned,” applications need boundaries within which they can run and be used securely. If you run UNIX or Linux systems, you may be familiar with creating chroot jails for applications. A chroot jail is the directory to which a program is confined. It changes the apparent top of the file system (the root of the file system) to be the directory for the current running process and any child processes. A program that is “rerooted” to another directory cannot access or name files outside that directory. Since this is a book on Microsoft attacks, we will turn our attention to DEP, a tool that goes much further than simply confining users to a defined space within a file system, in that it uses hardware and software to set boundaries for running applications in memory and prevent applications from using memory to which they were not originally assigned. Going back to “The Playground” analogy in the title for this layer, setting application boundaries and privileges through tools such as DEP is telling every kid where they can play and what parts of the playground are off limits.

According to Microsoft, DEP is “a set of hardware and software technologies that perform additional checks on memory to help prevent malicious code from running on a system…The primary benefit of DEP is to help prevent code execution from data pages. Typically, code is not executed from the default heap and the stack.”⁴ Note that DEP is not merely a security application that exists solely in the operating system. It is a system that is comprised of both hardware and software. The hardware portion of DEP, entitled hardware-enforced DEP, detects code that is running in memory, specifically the default heap and stack, and raises an exception when execution occurs. Software-enforced DEP prevents malicious code from exploiting built-in exception-handling within Windows.

DEP configuration for the system is controlled through a switch in the boot.ini file. The DEP settings can be configured through the System applet in the Control Panel. You must be logged on with administrator privileges to change this. The setting in boot.ini employs the following syntax: /noexecute=[policy_level], circled in the screenshot in Figure 2.7.

FIGURE 2.7. Selecting the System-Wide Policy Level for DEP in Boot.Ini

Because it is initialized during the boot process, DEP is enforced system-wide. There are four system-wide configurations for both hardware-enforced and software-enforced DEP.

1. OptIn (Default Setting): On systems with processors that can implement hardware-enforced DEP, DEP is enabled by default for limited system binaries and programs that “opt-in.” With this option, only Windows system binaries are covered by DEP by default.
2. OptOut: DEP is enabled by default for all processes. You can manually create a list of specific programs that do not have DEP applied by using the System dialog box shown in Figure 2.8.
3. AlwaysOn: This setting provides full DEP coverage for the whole system and all processes always run with DEP applied. With this option, the exceptions list shown in Figure 2.8 is not available.
4. AlwaysOff: This setting does not provide any DEP coverage for any part of the system, regardless of hardware DEP support.

alt1 Tip

Hardware-enforced and software-enforced DEP are configured in the same manner. If the system-wide DEP policy is set to OptIn, the same Windows core binaries and programs will be protected by both hardware-enforced and software-enforced DEP. If the system cannot use hardware-enforced DEP, the Windows core binaries and programs will be protected only by software-enforced DEP. Similarly, if the system-wide DEP policy is set to OptOut, programs that have been exempted from DEP protection will be exempted from both hardware-enforced and software-enforced DEP.

The process and applet for enabling DEP are same in Windows XP, Vista, and Windows 7. Open the System applet in the Control Panel. Click Advanced System Settings and either enter the administrator password or if you are using any edition of Windows Vista or Windows 7, confirm your assent to continue through User Account Control (UAC). Under the Performance tab, click Settings and click on the Data Execution Prevention tab. The open tab is displayed in Figure 2.8.

FIGURE 2.8. Changing DEP Settings

The default setting is that DEP is enabled for essential Windows programs and services only. Selecting the other radio button will enable DEP for all programs and services; you can manage what programs and services are excluded by using the Add… and Remove buttons. Clicking on the Add… button will bring up an Open File dialog box where you can select individual executable files to add to the list.

One note of caution: DEP has been and can be circumvented by skilled attackers. This is yet another defense-in-depth approach that should be taken to help prevent malicious attacks.