Chapter 7. Risk-Based Security Testing[1]
![Risk-Based Security TestingRisk-based security testingprocess overviewTouchpointslist ofrisk-based security testingParts of this chapter appeared in original form in IEEE Security & Privacy magazine co-authored with Bruce Potter [Potter and McGraw 2004].](http://imgdetail.ebookreading.net/software_development/21/0321356705/0321356705__software-security-building__0321356705__graphics__187fig01.gif)
A good threat is worth a thousand tests. | ||
| --BORIS BEIZER | ||
Security testing has recently moved beyond the realm of network port scanning to include probing software behavior as a critical aspect of system behavior (see the box From Outside→In to Inside→Out on page 189). Unfortunately, testing software security is a commonly misunderstood task. Security testing done properly goes much deeper than simple black box probing on the presentation layer (the sort performed by so-called application security tools, which I rant about in Chapter 1)—and even beyond the functional testing of security apparatus.
Testers must carry out a risk-based approach, grounded in both the system’s architectural reality and the attacker’s mindset, to gauge software security adequately. By identifying risks in the system and creating tests driven by those risks, a software security tester can properly focus on areas of code where an attack is likely to succeed. This approach provides a higher level of software security assurance than is possible with classical black box testing.
Security testing has much in common with (the new approach to) penetration testing as covered in Chapter 6. The main difference between security testing and penetration testing is the level of approach and the timing of the testing itself. Penetration testing is by definition an activity that happens once software is complete and installed in its operational environment. Also, by its nature, penetration testing is focused outside→in and is somewhat cursory. By contrast, security testing can be applied before the software is complete, at the unit level, in a testing environment with stubs and pre-integration.[2] Both approaches work best when they take risk analysis results, abuse cases, and functional security requirements into account.
Security testing should start at the feature or component/unit level, prior to system integration. Risk analysis carried out during the design phase (see Chapter 5) should identify and rank risks and discuss intercomponent assumptions. At the component level, risks to the component’s assets must be mitigated within the bounds of contextual assumptions. Tests should be structured in such a way as to attempt both unauthorized misuse of and access to target assets as well as violations of the assumptions the system writ large may be making relative to its components. A security fault may well surface in the complete system if tests like these are not devised and executed.
Security unit testing carries the benefit of breaking system security down into a number of discrete parts. Theoretically, if each component is implemented safely and fulfills intercomponent design criteria, the greater system should be in reasonable shape (though this problem is much harder than it may seem at first blush [Anderson 2001]).[3] By identifying and leveraging security goals during unit testing, the security posture of the entire system can be significantly improved.
Security testing should continue at the system level and should be directed at properties of the integrated software system. This is precisely where penetration testing meets security testing, in fact. Assuming that unit testing has successfully achieved its goals, system-level testing should shift the focus toward identifying intracomponent failures and assessing security risk inherent at the design level. If, for example, a component assumes that only another trusted component has access to its assets, a test should be structured to attempt direct access to that component from elsewhere. A successful test can undermine the assumptions of the system and would likely result in a direct, observable security compromise. Data flow diagrams, models, and intercomponent documentation created during the risk analysis stage can be a great help in identifying where component seams exist.
Finally, abuse cases developed earlier in the lifecycle (see Chapter 8) should be used to enhance a test plan with adversarial tests based on plausible abuse scenarios. Security testing involves as much black hat thinking as white hat thinking.
From Outside→In to Inside→Out
Traditional approaches to computer and network security testing focus on network infrastructure, firewalls, and port scanning. This is especially true of network penetration testing (see Chapter 6) and its distant cousin, software penetration testing (sometimes called application penetration testing by vendors). Early approaches to application-level penetration testing were lacking because they attempted to test all possible programs with a fixed number of (lame) canned attacks. Better penetration testing approaches take architectural risks, code scanning results, and security requirements into account, but still focus on an outside→in perspective.
The notion behind old-school security testing is to protect vulnerable systems (and software) from attack by identifying and defending a perimeter. In this paradigm, testing focuses on an outside→in approach.
One classic example is the use of port scanning with tools such as Nessus <http://www.nessus.org/> or nmap <http://www.insecure.org/nmap/> to probe network ports and see which service is listening. Figure 7-1 shows a classic outside→in paradigm focusing on firewall placement. In this figure, the LAN is separated from the Internet (or public network) by a firewall. The natural perimeter is the firewall itself, which is supposed to provide a choke point for network traffic and a position from which very basic packet-level enforcement is possible. Firewalls do things like “drop all packets to port 81” or “only allow traffic from specific IP addresses on specific ports through.”
Figure 7-1. The outside→in approach. A firewall protects a LAN by blocking various network traffic on its way in; outside→in security testing (especially penetration testing) involves probing the LAN with a port scanner to see which ports are “open” and which services are listening on those ports. A major security risk associated with this approach is that the services traditionally still available through the firewall are implemented with insecure software.
The problem is that this perimeter is only apparent at the network/packet level. At the level of software applications (especially geographically distributed applications), the perimeter has all but disappeared. That’s because firewalls have been configured (or misconfigured, depending on your perspective) to allow advanced applications to tunnel right through them.
A good example of this phenomenon is the SOAP protocol, which is designed (on purpose) to shuttle traffic through port 80 for various different applications. In some sense, SOAP is an anti-security device invented by software people so that they could avoid having to ask hard-nosed security people to open a firewall port for them. Once a tunnel like this is operational, the very idea of a firewall seems quaint.[*] In the brave new world of Service Oriented Architecture (SOA) for applications, we should not be surprised that the firewall is quickly becoming irrelevant.
By contrast, I advocate an inside→out approach to security, whereby software inside the LAN (and exposed on LAN boundaries) is itself subjected to rigorous risk management and security testing. This is just plain critical for modern distributed applications.
Software security is about making software behave in the presence of a malicious attack even though, in the real world, software failures usually happen spontaneously—that is, without intentional mischief [Leveson 1995]. Not surprisingly, standard software testing literature is only concerned with what happens when software fails, regardless of intent. The difference between software safety and software security is therefore the presence of an intelligent adversary bent on breaking the system. Most safety-critical systems (and high-assurance systems) posit a white hat world. Fact is, we live in a world with plenty of black hats as well, and we need to address that (head on).
Security is always relative to the information and services being protected, the skills and resources of adversaries, and the costs of potential assurance remedies; security is an exercise in risk management. Risk analysis, especially at the design level, can help us identify potential design-level security problems and their impact (see Chapter 5). Once identified and ranked, software risks can then help guide software security testing. Using a risk management framework, such as the RMF described in Chapter 2, allows us to track risks over time and thereby construct more relevant and more potent tests.
A vulnerability is an error that an attacker can exploit. Many types of vulnerabilities exist, and computer security researchers have created taxonomies of them, one of the first being Carl Landwehr [Landwehr, Bull, and McDermott 1993]. Vulnerabilities arise from defects, which in turn fall into two broad categories—implementation-level bugs and design-level flaws.
Attackers generally don’t care whether a vulnerability is due to a flaw or a bug, although bugs tend to be easier to exploit [Koziol et al. 2004]. Because attacks are now becoming more sophisticated, the notion of which vulnerabilities actually matter is changing. Although timing attacks, including the well-known race condition, were considered exotic just a few years ago, they’re common now [Bishop and Dilger 1996]. Similarly, two-stage buffer overflow attacks using trampolines were once the domain of software scientists but now appear in zero-day exploits [Hoglund and McGraw 2004]. On the horizon are arc injection attacks and other sophisticated control flow hacks [Pincus and Baker 2004]. I present a taxonomy of software security coding errors in Chapter 12. Thinking carefully about this taxonomy while developing a security test plan is a good tactic.
Design-level vulnerabilities are the hardest defect category to handle, but they’re also both prevalent and critical. Unfortunately, ascertaining whether a program has design-level vulnerabilities requires great expertise, which makes finding such flaws not only difficult but also particularly hard to automate. Even though finding flaws is a difficult undertaking, I cover it in some detail in Chapter 5.
Examples of design-level problems include error handling in object-oriented systems, object sharing and trust issues, unprotected data channels (both internal and external), incorrect or missing access control mechanisms, lack of auditing/logging or incorrect logging, and ordering and timing errors (especially in multithreaded systems). These sorts of flaws almost always lead to security risk.
Software security practitioners perform many different tasks to manage software security risks, such as:
Creating security abuse/misuse cases
Listing normative security requirements (and security features and functions)
Performing architectural risk analysis
Building risk-based security test plans
Wielding static analysis tools
Performing security tests
Performing penetration testing in the final environment
Cleaning up after security breaches
Three of these practices are particularly closely linked—architectural risk analysis, risk-based security test planning, and security testing—because a critical aspect of security testing relies on directly probing security risks. Chapter 5 explains how to approach a software security risk analysis, the end product being a set of security-related risks ranked by business or mission impact. Chapter 2 explains how to keep track of security risks and properly manage them over time in an RMF.
The pithy aphorism “Software security is not security software” provides an important motivator for security testing. Although security features, such as cryptography, strong authentication, and access control, play a critical role in software security, security itself is an emergent property of the entire system, not just the security mechanisms and features. A buffer overflow is a security problem regardless of whether it exists in a security feature or in the noncritical GUI.
For this reason, security testing must necessarily involve two diverse approaches:
Functional security testing: testing security mechanisms to ensure that their functionality is properly implemented
Adversarial security testing: performing risk-based security testing motivated by understanding and simulating the attacker’s approach
Together, these two distinct activities are a mix of white hat (security functionality) and black hat (security attack) philosophies. Security testing must mix both approaches or it will fail to cover critical areas.
Many developers erroneously believe that security involves only the liberal application and use of various security features, which leads to the incorrect belief that “adding SSL” is tantamount to securing an application. Software security practitioners bemoan the over-reliance on “magic crypto fairy dust” as a reaction to this problem. Software testers charged with security testing often fall prey to the same thinking.
It’s not that we shouldn’t test the crypto fairy dust to determine its potency. It’s just that most security attacks ignore the security mechanisms in favor of looking for software defects anywhere in the system. Security testing needs to cover the attacker’s mindset just as well as it covers security functionality.
Like any other form of testing, security testing involves determining who should do the testing and what activities they should undertake.
Because security testing involves two approaches, the question of who should do it has two answers. Standard testing organizations using a traditional approach can perform functional security testing. For example, ensuring that access control mechanisms work as advertised is a classic functional testing exercise. Since we basically know how the software should behave, we can run some tests and make sure that it does.[4]
On the other hand, traditional QA staff will have more difficulty performing risk-based security testing. The problem is one of expertise. First, security tests (especially those resulting in complete exploit) are difficult to craft because the designer must think like an attacker. Second, security tests don’t often cause direct security exploit and thus present an observability problem. Unlike in the movies, a security compromise does not usually result in a red blinking screen flashing the words “Full Access Granted.” A security test could result in an unanticipated outcome that requires the tester to perform further sophisticated analysis. Bottom line: Risk-based security testing relies more on expertise and experience than we would like—and not testing experience, security experience.
The software security field is maturing rapidly. I hope we can solve the experience problem by identifying best practices, gathering and categorizing knowledge, and embracing risk management as a critical software philosophy.[5] At the same time, academics are beginning to teach the next generation of builders a bit more about security so that we no longer build broken stuff that surprises us when it is spectacularly exploited.
Books, such as How to Break Software Security and Exploiting Software, help educate testing professionals on how to think like an attacker during testing [Whittaker and Thompson 2003; Hoglund and McGraw 2004]. Nevertheless, software exploits are surprisingly sophisticated these days, and the level of discourse found in books and articles is only now coming into alignment.
White and black box testing and analysis methods both attempt to understand software, but they use different approaches depending on whether the analyst or tester has access to source code. White box analysis involves analyzing and understanding both source code and the design. This kind of testing is typically very effective in finding programming errors (bugs when automatically scanning code and flaws when doing risk analysis); in some cases, this approach amounts to pattern matching and can even be automated with a static analyzer (the subject of Chapter 4). One drawback to this kind of testing is that tools might report a potential vulnerability where none actually exists (a false positive). Nevertheless, using static analysis methods on source code is a good technique for analyzing certain kinds of software. Similarly, risk analysis is a white box approach based on a thorough understanding of software architecture.
Black box analysis refers to analyzing a running program by probing it with various inputs. This kind of testing requires only a running program and doesn’t use source code analysis of any kind. In the security paradigm, malicious input can be supplied to the program in an effort to break it: if the program breaks during a particular test, then we might have discovered a security problem. Black box testing is possible even without access to binary code—that is, a program can be tested remotely over a network. If the tester can supply the proper input (and observe the test’s effect), then black box testing is possible.
Any testing method can reveal possible software risks and potential exploits. One problem with almost all kinds of security testing (regardless of whether it’s black or white box) is the lack of it—most QA organizations focus on features and spend very little time understanding or probing nonfunctional security risks. Exacerbating the problem, the QA process is often broken in many commercial software houses due to time and budget constraints and the belief that QA is not an essential part of software development.
Case studies can help make sense of the way security testing can be driven by risk analysis results. See the box An Example: Java Card Security Testing.
Coder’s Corner
Let’s take a look at one of the tests that we built for Java Card security testing. This test set as a whole probes whether shareable objects behave properly on a card.
First, the interface specification:
package tests.config1.jcre.JcreTest010_1;
import javacard.framework.Shareable;
import ssg.framework.*;
public interface shareableInterface extends Shareable {
public void shareObject();
}This little glob of code implements the shared interface and sets up the test harness.
package tests.config1.jcre.JcreTest010_1; import javacard.framework.*; import ssg.framework.*; public class JcreTest010_1a extends Applet implementsshareableInterface { static byte[] shareableObjectBuffer; private JcreTest010_1a() { shareableObjectBuffer = new byte[10]; for(byte i=0; i < 10; i++) shareableObjectBuffer[i] = 0x11; register(); } public void shareObject() { for(byte i=0; i < 10; i++) shareableObjectBuffer[i] = 0x22; } public void testFunc() { for(byte i=0; i < 10; i++) shareableObjectBuffer[i] = 0x33; } public Shareable getShareableInterfaceObject(AID client_aid,
Then we can run tests like this. (I show you only one of the five tests related to shareable interfaces just to keep things simple.)
package tests.config1.jcre.JcreTest010_2;
import javacard.framework.*;
import ssg.framework.*;
import tests.config1.jcre.JcreTest010_1.*;
public class JcreTest010_2a extends Applet
{
byte [] serverAID = {74,99,114,101,84,101,115,116,48,49,48,49,97};
byte [] AIDValue;
private JcreTest010_2a()
{
AIDValue = new byte[16];
register();
}
public static void install( byte[] bArray, short bOffset, byte
bLength )
{
new JcreTest010_2a();
}
public void process(APDU apdu)
{
AID serverAIDObject = JCSystem.lookupAID(serverAID, (short)0,
(byte)serverAID.length);
if(serverAIDObject == null)
ISOException.throwIt(ISO7816.SW_WRONG_P1P2); // 0x6B00
if((serverAIDObject.equals(serverAID, (short)0,
(byte)serverAID.length)) ==
false)
ISOException.throwIt(ISO7816.SW_CORRECT_LENGTH_00); // 0x6C00
shareableInterface sio = (shareableInterface)
(JCSystem.getAppletShareableInterfaceObject(serverAIDObject,
(byte)0));
if(sio == null) {
byte length = serverAIDObject.getBytes(AIDValue, (short)0);
byte[] apdu_buffer = apdu.getBuffer();
apdu.setOutgoing();
apdu.setOutgoingLength((short)length);
Util.arrayCopy(AIDValue, (short)0, apdu_buffer, (short)0,
(short)length);
apdu.sendBytes((short)0, (short)length);
ISOException.throwIt(ISO7816.SW_INS_NOT_SUPPORTED); // 0x6D00
}
sio.shareObject();
}
}What we found in practice on one of the many real cards we tested was that the shareable interface tests all worked fine. What failed was the test teardown procedure that tries to leave the card in the same state as when we started. When this failed, we did some investigation by hand and uncovered some interesting issues.
There is no silver bullet for software security; even a reasonable security testing regimen is just a start. Unfortunately, security continues to be sold as a product, and most defensive mechanisms on the market do little to address the heart of the problem, which is bad software. Instead, they operate in a reactive mode: Don’t allow packets to this or that port, watch out for files that include this pattern in them, throw partial packets and oversized packets away without looking at them. Network traffic is not the best way to approach the software security predicament because the software that processes the packets is the problem. By using a risk-based approach to software security testing, testing professionals can help solve security problems while software is still in production.
Of course, any testing approach is deeply impacted by software process issues. Because of eXtreme Programming’s (XP) “test first” philosophy, adopting a risk-based approach may be difficult if you are in an XP shop. See the following box, eXtreme Programming and Security Testing.
Put simply, the biggest problems in software security exist because software takes input (see the taxonomy of coding errors in Chapter 12). Whether to trust input (including the very format that the input takes) is a critical question that all software builders must ponder.
Exploiting Software is filled with examples of programs that break when malformed or maliciously formed input leads to security compromise [Hoglund and McGraw 2004]. From the much-ballyhooed buffer overflow (which involves putting too much input in too small a place) to the likewise overhyped SQL injection attack and cross-site scripting (XSS) attacks, trusting input turns out to be the common root cause.
Carefully handling input is paramount to software security. Note that input includes things like register settings, environment variables, file contents, and even network configuration. If your program consumes data from “out there,” you need to think carefully about who can dink around with the stuff your program eats.
Attacker toolkits (briefly described in Chapter 6) focus plenty of attention on input, with a plethora of fault injection tools, grammar generators, re-players, and the like. By its very nature, penetration testing is obsessed with input as well (mostly because crafting malicious input is the main way to break a system from the outside). If your program accepts input over the network, it needs to be very skeptical of what it is getting.
Using a black-list approach (which tries to enumerate all possible bad input) is silly and will not work. Instead, software needs to defend its input space with a white-list approach (and a Draconian white-list approach, for that matter). If your program enforces statements like “Accept only input of 32-bits as an Integer” (something that is easy to do in a modern type-safe language), you’re better off right off the bat than with a system that accepts anything but tries to filter out return characters. Make sure that your testing approach delves directly into the black-list/white-list input-filtering issue.
Microsoft pays plenty of attention to malicious input in its approach to software security. You should too. (See Writing Secure Code [Howard and LeBlanc 2003].)
Don’t get too caught up in solving only the input problem. Testing around malicious input is a necessary but not sufficient condition. Security testing needs to get past input myopia by focusing on data structures, components, APIs, program state, and so on.
The forest-level view created during architectural risk analysis (see Chapter 5) is very useful in planning security testing. In addition to building tests around risks that remain in the system, testers should consider things like:
Sockets
Pipes
The Win32 Registry
Files
Remote procedure calls (RPCs)
Command-line arguments
And so on
Time is a critical issue to think about in modern software systems. There are two major aspects of time to consider. The first has to do with program state and state preservation. Because some modern software protocols in common use (like HTTP) are stateless, a variety of hacks and kludges around the state preservation problem have been devised. Many of these kludges are inherently insecure. Security testers must consider what happens when state is changed by an attacker. This can be as simple as changing a “hidden” variable in a URL or as complex as de-serializing an object, manipulating it, and re-serializing it.
The second aspect of time that is essential to think about is related to state, but only indirectly. When multiple processes interact and share some kind of data structure (either by querying the environment or by using locks and semaphores), a new line of attack is opened up in the form of changing the environment that is being queried or otherwise messing around with locks. Time-of-check–time-of-use (TOCTOU) race conditions are always worth considering when testing a multithreaded system. Even more subtle data races are also an important and often overlooked category of errors to consider.
One problem is that most developers are unfamiliar with the effects of multithreading on their systems. That means they often overlook subtle time-based attacks. I believe that timing attacks (both data races and starvation attacks) are a future attack category that will be much more commonly encountered than they are now. We’ve begun to see hints of this already (for more, see the taxonomy in Chapter 12).
Getting inside a program and thinking about control flow and data flow is an excellent strategy for devising a solid testing regimen. Penetration testing, because of its outside→in bias, only begins to scratch the surface of an inside-the-software testing approach. Security testing goes beyond penetration testing by adopting a clear inside→out approach focused on software guts.
Books like The Shellcoder’s Handbook, How to Break Software Security, and Exploiting Software help software professionals understand the mind of the attacker and the kinds of program understanding tools commonly used by attackers [Koziol et al. 2004; Whittaker and Thompson 2003; Hoglund and McGraw 2004]. This is a critical undertaking for security testers. Unless a security tester thinks like a bad guy (black hat firmly on head), security testing will not be effective.
Software is so broken today that simple penetration testing usually works. Getting past the obvious is only necessary when the low-hanging fruit discovered during simple penetration testing is taken care of. Then things get tricky fast. Be prepared for things to get tricky. Then plan to adopt risk-based security testing.
[1] Parts of this chapter appeared in original form in IEEE Security & Privacy magazine co-authored with Bruce Potter [Potter and McGraw 2004].
[2] This distinction is similar to the slippery distinction between unit testing and system testing.
[3] Ross Anderson refers to the idea of component-based distributed software and the composition problem as “programming the devil’s computer.”
[*] If you’re having trouble getting through the firewall, just aim your messages through port 80, use a little SOAP, and you’re back in business.
[4] This is not to trivialize the critical field of software testing. Testing is a difficult and painstaking activity that requires years of experience to do right.
[5] The three pillars of software security.