Trusting user and parameter input is a frequent source of security problems. Attacks that take advantage of little to no input validation include cross-site scripting, illegal pointer values, integer overflows, and DNS cache poisoning (refer to the glossary for definitions of these types of attacks). In addition, inadequate input validation can lead to buffer overflows and SQL defects as described below. All of these types of attacks can pose risks to confidentiality and integrity. One of the more effective approaches for input validation is to use a whitelist, which lists all known good inputs that a system is permitted to accept and excludes everything else (including characters used to perform each type of attack).

Exceptions are events that disrupt the normal flow of code. Programming languages may use a mechanism called an exception handler to deal with unexpected events such a divide-by-zero attempt, violation of memory protection, or a floating-point arithmetic error. Such exceptions could be handled by the code by checking for conditions that can lead to such violations. When such checks are not made, however, exception handling passes control from the function with that error to a higher execution context in an attempt to recover from that condition. Such exception handling disrupts the normal flow of the code. The security concerns that arise from exception handling are discussed in [McGraw 2006].

Buffer overflows are a leading method used to exploit software by remotely injecting malicious code into a target application [Hoglund 2004; Viega 2001]. The root cause of buffer overflow problems is that commonly used programming languages such as C and C++ are inherently unsafe. No bounds checks on array and pointer references are carried out, meaning that a developer must check the bounds (an activity that is often overlooked) or risk encountering problems.

When writing to buffers, C/C++ programmers must take care not to store more data in the buffer than it can hold. When a program writes past the bounds of a buffer, a buffer overflow occurs and the next contiguous chunk of memory is overwritten. C and C++ allow programs to overflow buffers at will. No runtime checks are performed that might prevent writing past the end of a buffer, so developers have to perform the checks in their own code.

Reading or writing past the end of a buffer can cause a number of diverse (and often unanticipated) behaviors: (1) Programs can act in strange ways, (2) programs can fail completely, and (2) programs can proceed without any noticeable difference in execution. The side effects of overrunning a buffer depend on the following issues:

The indeterminate behavior of programs that have overrun a buffer makes them particularly tricky to debug. In the worst cases, a program may overflow a buffer and not show any adverse side effects at all. As a result, buffer overflow problems often remain invisible during standard testing. The important thing to realize about buffer overflows is that any data that happens to be allocated near the buffer can potentially be modified when the overflow occurs.

Memory usage vulnerabilities will continue to be a fruitful resource for exploiting software until languages that incorporate memory management schemes enter into wider use.

SQL injection is currently the principal technique used by attackers to take advantage of nonvalidated input defects to pass SQL commands through an application for execution by a database. The security model used by many applications assumes that a SQL query is a trusted command. In this case, the defect lies in the software’s construction of a dynamic SQL statement based on user input.

Attackers take advantage of the fact that developers often chain together SQL commands with user-provided parameters, meaning that the attackers can, therefore, embed SQL commands inside these parameters. As a result, the attacker can execute arbitrary SQL queries and/or commands on the database server through the application. This ability enables attackers to exploit SQL queries to circumvent access controls, authentication, and authorization checks. In some instances, SQL queries may allow access to commands at the level of the host operating system. This can be done using stored procedures.

Race conditions take on many forms but can be characterized as scheduling dependencies between multiple threads that are not properly synchronized, causing an undesirable timing of events. An example of a race condition that could have a negative outcome on security is when a specific sequence of events is required between Event A and Event B, but a race occurs and the proper sequence is not ensured by the software program. Developers can use a number of programming constructs to control the synchronization of threads, such as semaphores, mutexes, and critical sections. Race conditions fall into three main categories:

Additional security concerns that arise from these and other types of software vulnerabilities are discussed in [McGraw 2006].

Static source code analysis is the process by which software developers check their code for problems and inconsistencies before compiling it. Developers can automate the analysis of source code by using static analysis tools. These tools scan the source code and automatically detect errors that typically pass through compilers and can cause problems later in the SDLC.

Many modern static analysis tools generate reports that graphically present the analysis results and recommend potential resolutions to identified problems.

Identifying security vulnerabilities is complicated by the fact that they often appear in hard-to-produce software states or crop up in unusual circumstances. Static analysis has the advantage of being performed before a program reaches a level of completion where dynamic analysis or other types of analysis can be meaningfully used. However, static code analyzers should not be viewed as a panacea to all potential problems. These tools can produce false positives and false negatives, so their results should be taken with the proverbial “grain of salt.” That is, results indicating that zero security defects were found should not be taken to mean that your code is completely free of vulnerabilities or 100 percent secure; rather, these results simply mean that your code has none of the patterns found in the analysis tool’s rulebase for security defects.

The greatest promise of static analysis tools derives from their ability to automatically identify many common coding problems. Unfortunately, implementation bugs created by developer errors are often only part of the problem. Static analysis tools cannot evaluate design and architectural flaws. They cannot identify poorly designed cryptographic libraries or improperly selected algorithms, and they cannot point out design problems that might cause confusion between authentication and authorization. They also cannot identify passwords or magic numbers embedded in code. One further drawback to automated code analysis is that the tools are prone to producing false positives when a potential vulnerability does not exist. This is especially true of older freeware tools, most of which are not actively supported; many analysts do not find these tools to be useful when analyzing real-world software systems.^[3] Commercial tool vendors are actively addressing the problem of false positives and have made considerable progress in this realm, but much remains to be done.

Static code analysis can be used to discover subtle and elusive implementation errors before the software is tested or placed into operation. By correcting subtle errors in the code early, project managers can reduce testing efforts and minimize operations and maintenance costs. Static code analysis tools can be applied in a variety of ways, all of which lead to higher-quality software. This said, static analysis tools can identify only a subset of the vulnerabilities leading to security problems. These tools must always be used in conjunction with manual analysis and other software assurance methods to reduce vulnerabilities that cannot be identified based on patterns and rules.

Metric analysis produces a quantitative measure of the degree to which the analyzed code possesses a given attribute. An attribute is a characteristic or a property of the code. For example,

The process of using code metrics begins by deriving metrics that are appropriate for the code under review. Then data is collected, and metrics are computed and compared to preestablished guidelines and historical data (such as the number of defects per 1000 lines of code). The results of these comparisons are used to analyze the code with the intent of improving the measured qualities.

Two classes of quantitative software metrics are distinguished: absolute and relative. Absolute metrics are numerical values that represent a characteristic of the code, such as the probability of failure, the number of references to a particular variable in an application, or the number of lines of code. Absolute metrics do not involve uncertainty. There can be one and only one correct numerical representation of a given absolute metric. In contrast, relative metrics provide a numeric representation of an attribute that cannot be precisely measured, such as the degree of difficulty in testing for buffer overflows. There is no objective, absolute way to measure such an attribute. Multiple variables are factored into an estimation of the degree of testing difficulty, and any numeric representation is just an approximation.

The BSI Web site provides a number of code analysis process flow diagrams for source code review, static code analysis, and metric analysis, as well as for dynamic analysis, fault injection, cryptanalysis, and random-number generator analysis. We encourage you to consult the Web site [BSI 19] and [McGraw 2006] for further details.

Two common methods for testing whether software has met its security requirements are functional security testing and risk-based security testing [McGraw 2006]. Functional testing is meant to ensure that software behaves as specified and so is largely based on demonstrating that requirements defined in advance during requirements engineering (see Chapter 3) are satisfied at an acceptable level. Risk-based testing probes specific risks that have been identified through risk analysis. The next two sections discuss how functional and risk-based testing can be used to enhance confidence in the software’s security.

Software engineers may not understand how to implement some security requirements. In one example, a Web application was found to be vulnerable to a directory traversal attack, where a URL containing the string “..” was used to access directories that were supposedly forbidden to remote clients. To counter this possibility, developers used a blacklist technique, in which a list is created and used to exclude or filter out bad input data and bad characters. URLs that contained this string were added to the blacklist and thus disallowed. However, blacklists are not infallible:

Testing cannot demonstrate the absence of software problems; it can only demonstrate (sometimes) that problems are present [Dijkstra 1970]. The problem is that testers can try out only a limited number of test cases; the software might work correctly for those cases and fail for other cases. Therefore, testing a mitigation measure is not enough to guarantee that the corresponding risk has truly been eliminated, and this caveat is especially important to keep in mind when the risk in question is a severe one.

Also, when bugs are fixed, the fix is sometimes not subjected to the same scrutiny as those features that were part of the original software design. For example, a problem that should normally be detected in design reviews might slip through the cracks if it shows up as part of a bug fix. Sometimes software that has been repaired is retested simply by running the original test suite again—but that approach works poorly for the caveats described here.

Functional testing is meant to probe whether software behaves as it should, but so far we have focused only on requirements-based testing. A number of other functional testing techniques (as described in 170 on pages 170–171) do not rely on defined requirements. These techniques are described in more detail in [BSI 21].

As a basis for defining test conditions, past experience comes into play in two ways. First, a mature test organization typically has a set of test templates that outline the test techniques to be used for testing against specific risks and requirements in specific types of software modules. Test templates are usually created during testing projects, and they accumulate over time to capture the organization’s past experience. This book does not provide test templates, but attack patterns appropriate for this purpose are described in several other sources [Hoglund 2004; Whittaker 2002, 2003].

Another way to derive test scenarios from past experience is to use incident reports. Incident reports can simply be bug reports, but in the context of security testing they can also be forensic descriptions of successful intruder activity. Furthermore, vulnerability reports are often followed by proofs of concept to demonstrate how the reported vulnerability can be exploited. Sometimes these proofs of concept are actual exploits; at other times they simply show that a vulnerability is likely to be exploitable. For example, if a buffer overflow can be made to cause a crash, then it can usually be exploited by an attacker as well. Sometimes it is sufficient to find evidence of vulnerabilities as opposed to actual exploits, so that the resulting proofs of concept can be used as the basis for test scenarios. When devising risk-based tests, it can be useful to consult IT security personnel, as their jobs involve keeping up-to-date on vulnerabilities, incident reports, and security threats.

Attack patterns (as discussed in Chapter 2) can be used effectively during software security testing to craft test cases that reflect attacker behavior and to help identify test cases that validate secure behavior.

Finally, threat modeling can be leveraged to help create risk-based tests. For example, if inexperienced intruders (e.g., script kiddies) are expected to pose a major threat, then it might be appropriate to probe the software under test with automated tools; intruders often use the same tools (see the description of black-box testing in Section 5.5.4).

Additional thought processes that might be helpful in creating new tests for negative requirements include (1) understanding a software component and its environment, (2) understanding the assumptions of the developers, and (3) building a fault model (hypotheses about what might go wrong). Consult [BSI 21] for further details.

One popular approach to system testing is black-box testing. Black-box testing uses methods that do not require access to source code. Either the test engineer does not have access or the details of the source code are irrelevant to the properties being tested. As a consequence, black-box testing focuses on the externally visible behavior of the software, such as requirements, protocol specifications, APIs, or even attempted attacks. Within the security test arena, black-box testing is normally associated with activities that occur during the pre-deployment test phase (system test) or on a periodic basis after the system has been deployed.

Black-box test activities almost universally involve the use of tools, which typically focus on specific areas such as network security, database security, security subsystems, and Web application security. For example, network security tools include port scanners to identify all active devices connected to the network, services operating on systems connected to the network, and applications running for each identified service. Vulnerability scanning tools identify specific security vulnerabilities associated with the scanned system based on information contained within a vulnerability database. Potential vulnerabilities include those related to open ports that allow access to insecure services, protocol-based vulnerabilities, and vulnerabilities resulting from poor implementation or configuration of an operating system or application.

For more information on black-box testing and test tools, refer to [BSI 23].

Another common approach for conducting certain aspects of system security testing is penetration testing, which allows project managers to assess how an attacker is likely to try to subvert a system. At a basic level, the term “penetration testing” refers to testing the security of a computer system and/or software application by attempting to compromise its security—in particular, the security of the underlying operating system and network component configurations.

Conventional penetration testing tools come in a variety of forms, depending on which sort of testing they can perform. A key distinguishing factor is the perspective from which each type of tool operates—that is, whether a testing tool evaluates its target from afar or from relatively close up (i.e., at least within the same computer system). Popular classes of tools used in penetration testing today include host-based, network-based, and application scanning [Fyodor 2006].

For example, most organizations, when doing network-based penetration testing, follow a process that looks something like this (Steps 1–3 constitute the vulnerability scanning approach mentioned earlier):

For more information on penetration testing and pitfalls to avoid, refer to [BSI 24]. For more information on penetration testing tools, refer to [BSI 25].