Chapter 11. Knowledge for Software Security[1]
Knowledge is power. | ||
| --FRANCIS BACON | ||
Knowledge management can play a central role in encapsulating and spreading the emerging discipline of software security more efficiently. This chapter is about the kinds of security knowledge that can be used to provide a solid foundation for software security practices.
Knowledge is more than simply a list of things we know or a collection of facts. Simply put, information and knowledge aren’t the same thing, and it is important to understand the difference. Knowledge is information in context—information put to work using processes and procedures. A checklist of potential security bugs in C and C++ is information; the same information built into a static analysis tool is knowledge.
At this nascent stage of the game in software security, a number of early adopters have created various checklists for use when thinking about software security and application security. One of the problems with these lists is that they have a tendency to combine categories of information in hard-to-grok ways. For example, a ““Top Ten Things to Know about Application Security”” document that treats “Apply the Principle of Least Privilege” the same as “Avoid Buffer Overflows,” “Monitor BugTraq,” and “Use a Code Scanning Tool” combines lots of good ideas in an incoherent package. It is better to organize software security knowledge into coherent chunks.
The first hurdle along these lines to overcome is the propensity to think of software security as a coding issue. I like to refer to this kind of approach as the “bug parade.” Sure, there are hundreds of bugs that can lead to security problems (especially in languages like C and C++). But simply developing a checklist of coding issues to avoid in C and having your developers read it will not solve the software security problem.[2] If instead of making a static list, we build a database of coding issues and create a tool to help us uncover these problems, then we’re getting somewhere. This is precisely what is happening with the static analysis space.
Of course, by now we know that we must address bugs (of the sort that a tool can easily find) and flaws (which require a smart human to find). We ignore flaws by declaring them “too hard to deal with at this time” at our peril.
The second hurdle is the incorrect belief that software security is really about adopting various security features and/or conventions. One place where this is going particularly wrong is in the creation of generic classes for filtering input. We all know by now that a black-list solution to the input-filtering problem (trying to identify all possible malicious input) is inferior to a white-list solution (ensuring that only input that plays by certain rules is allowed). The problem is that black lists are potentially infinite every time—there is no way to anticipate future malicious input. Consider for a moment the encoding problem and various Unicode attacks discussed in Exploiting Software and you’ll see what I mean [Hoglund and McGraw 2004].
Given a thorough understanding of a program (say, when you’re building it), you are in a perfect position to create a correct input-filtering approach since you know precisely what kind of input you are expecting. Wrongheaded thinking has led to the idea of “security classes” that you can buy and link into your code. In the case of generic filtering capability, this idea is unlikely to work. Of course, there is nothing wrong with adopting great coding practices and even borrowing solid code to use. In any event, as this book demonstrates, software security is more about assurance than it is about features. Some people call the feature-based approach to software security a “cookbook” approach. Cookbooks can certainly help you with recipes, but just reading cookbooks without ever turning on your stove and actually tasting stuff won’t make you a good cook. Experience is the most powerful teacher.
The third and final major hurdle is overuse of the checklist. Checklists are great in the hands of an expert. They serve as reminders of things to think about. However, checklists are by their very definition incomplete. Consider the STRIDE model from Writing Secure Code [Howard and LeBlanc 2003]. The activity of thinking carefully about Spoofing, Tampering, Repudiation, Information disclosure, Denial of service, and Elevation of privilege while you ponder system security is a great idea. The problem is that there are definitely more than six categories of attack. If you limit your thinking to a checklist, you will likely overlook interesting risks that lead to new attacks. Attackers know this well, and they will go out of their way to game this problem. For example, no virus writer worth his salt will release a new virus without first running every available commercial antivirus checker against it as an acceptance test. (Not to imply that there aren’t plenty of really dumb virus writers out there.)
The way around these hurdles is to organize and apply software security knowledge with care.
This chapter may be too academic or research oriented for some. Software security practitioners and software security scientists will certainly want to develop the catalogs we cite (or participate in group exercises to develop a common set of open catalogs for all). But large organizations worried about adopting software security programs (as described in Chapter 10) will be better served with the information architecture covered there. This chapter is more about the intellectual exercise of organizing and cataloging knowledge than it is about making that knowledge actionable in an enterprise.
Software developers place a high premium on knowledge. Experience is king, and expertise is very valuable. The software field is in a perpetual state of change, and keeping on top of all possible new technologies is very difficult, if not impossible. Developers show great respect for those who master aspects of the expanding field and are able to help bring others along. This is the kind of phenomenon that drives topnotch developer conferences like SD West and SD Best Practices (called SD East by most people)—find both here <http://www.sdexpo.com/>.
Similarly, software security practitioners place a premium on knowledge and experience. In a field where most practitioners are still being exposed to the basics (think checklists and basic coding rules), the value of master craftsmen who have “been there and done that,” learned a number of lessons the hard way, and are able to transfer that experience to others is very high.
The bad news is that there aren’t enough master craftsmen in software security to apprentice and train all software developers, software architects, and software security newbies effectively. The good news is that critical software security knowledge and expertise can be compiled from those in the know and then shared widely. This possibility yields a potentially higher return than the pervasive one-to-one method of apprenticeship practiced today. Through the aggregation of knowledge from a number of experienced craftsmen, knowledge management can provide a new software security practitioner access to the knowledge and expertise of all the masters, not just one or two.
Software security knowledge is multifaceted and can be applied in diverse ways. As the software lifecycle unfolds, security knowledge can be directly and dynamically applied through the use of knowledge-intensive best practices like the touchpoints in this book. During professional training and resource development, security knowledge can be drawn on for pedagogical application, sparking stories and anecdotes. During academic training, security knowledge can inform basic coding and design curricula. All of these activities are beginning to happen in software security. For this reason, a sophisticated knowledge management approach is necessary.
Security knowledge can be organized according to the taxonomy introduced in the box Software Security Unified Knowledge Architecture. Seven knowledge catalogs (principles, guidelines, rules, vulnerabilities, exploits, attack patterns, and historical risks) are grouped into three knowledge categories (prescriptive knowledge, diagnostic knowledge, and historical knowledge).
Two of the seven catalogs are likely to be familiar to software developers with only a passing familiarity with software security—vulnerabilities and exploits. These catalogs have been in common use for quite some time and have even resulted in collection and cataloging efforts serving the security community.[3] Similarly, principles—stemming from the seminal work of Saltzer and Schroeder [1975]—and rules—identified and captured in static analysis tools, such as ITS4 (see Appendix B)—are fairly well understood. Knowledge catalogs only more recently identified include guidelines (often built into prescriptive frameworks for technologies such as .NET and J2EE), attack patterns [Hoglund and McGraw 2004], and historical risks. Together, these various knowledge catalogs provide a basic foundation for a unified knowledge architecture supporting software security.
Software Security Unified Knowledge Architecture
Figure 11-1 shows a basic software security knowledge schema relating the seven catalogs.
Figure 11-1. The basic schema displayed here shows one way to organize and interrelate software security knowledge. There are seven distinct knowledge catalogs, which can be divided into three knowledge categories.
The category prescriptive knowledge includes three knowledge catalogs: principles, guidelines, and rules. These sets span a continuum of abstraction from high-level architectural principles at the level of philosophy (e.g., the principle of least privilege [Saltzer and Schroeder 1975]) to very specific and tactical code-level rules (e.g., avoid the use of the library function gets() in C). Guidelines fall somewhere in the middle of this continuum (e.g., make all Java objects and classes final(), unless there’s a good reason not to [McGraw and Felten 1999]). As a whole, the prescriptive knowledge category offers advice for what to do and what to avoid when building secure software.
The category diagnostic knowledge includes three knowledge catalogs: attack patterns, exploits, and vulnerabilities. Rather than prescriptive statements of practice, diagnostic knowledge helps practitioners (including operations people) recognize and deal with common problems that lead to security attack. Vulnerability knowledge includes descriptions of software vulnerabilities experienced and reported in real systems (often with a bias toward operations). Exploits describe how instances of vulnerabilities are leveraged into particular security compromise for particular systems. Attack patterns describe common sets of exploits in a more abstract form that can be applied across multiple systems. Such diagnostic knowledge is particularly useful in the hands of a security analyst, though its value as a resource to be applied during development is considerable (e.g., consider the utility of attack patterns to abuse case development).
The category historical knowledge includes the knowledge catalog historical risks and, in some cases, vulnerabilities (e.g., the collection in the CVE <http://www.cve.mitre.org/>). Rather than derivations or abstractions, this catalog represents detailed descriptions of specific issues uncovered in real-world software development efforts and must include a statement of impact on the business or mission proposition. As a resource, this knowledge offers tremendous value in helping to identify similar issues in new software efforts without starting from scratch. It also provides a continuing source for identifying new instances of other knowledge catalogs described here: principles, guidelines, rules, vulnerabilities, and attack patterns.
Table 11-1 provides a bird’s-eye view of each knowledge catalog. Each entry includes a brief description, a sample schema for tracking instances, and a short list of software artifacts (arising from most software lifecycles) that the knowledge impacts the most. The idea here is to create a number of inter-related catalogs for use throughout the software lifecycle.
Table 11-1. A Bird’s-Eye View of Software Security Knowledge Catalogs
Description | Sample High-Level Schema | Relevant SDLC Artifacts | |
|---|---|---|---|
Principles | A principle is a statement of general security wisdom derived from experience. Though principles exist at the level of philosophy, they stem from real-world experience building secure systems. Principles are useful for both diagnosing architectural flaws in software and practicing good security engineering. |
|
|
Guidelines | A guideline is a recommendation for things to do or to avoid during software development, described at the semantic level. Guidelines exist for a specific technical context (e.g., J2EE, .NET, Linux kernel module, and so on). Guidelines are best enforced and evaluated through human analysis. Guidelines can help uncover both architectural flaws and implementation bugs. |
|
|
A rule is a recommendation for things to do or to avoid during software development, described at the level of syntax. A rule can be verified through lexical scanning or constructive parsing of software (source or binary). Rules exist for specific programming languages (e.g., C, C++, PHP, Java, and so on). Rules can help uncover implementation bugs. |
|
| |
Attack patterns | An attack pattern is a generalized pattern developed by reasoning over large sets of software exploits. Attack patterns are useful for identifying and qualifying the risk that a given exploit will occur in a software system. They are also useful in designing misuse and abuse cases and specific security tests. |
|
|
A historical risk is a risk identified in the course of an actual software development effort. At its core, a risk is a pairing of a condition/event with a quantification of the likelihood that it will occur and a quantification of the impact it will have. Historical risks are good resources for early identification of potential issues in a software development effort, for potential clues to effective mitigations, and for improvements to the consistency and quality of risk management in the software development process. |
|
| |
Vulnerabilities | A vulnerability is the result of a defect in software that can be used by an attacker to gain access to or negatively affect the security of a computer system. |
|
|
An exploit is a particular instance of an attack on a computer system that leverages a specific vulnerability or set of vulnerabilities. |
|
|
Work on fleshing out the knowledge catalogs identified here has been underway for some time by various groups. Makers of static analysis tools have pushed the envelope when it comes to rules, for example, while work sponsored by the Department of Homeland Security (and carried out by Cigital and SEI) has focused on principles and guidelines. The results of these efforts are available on the Web at <http://buildsecurityin.us-cert.gov/portal/> and should prove very useful for software security practitioners. More on the DHS effort can be found later in this chapter.
Software security knowledge can be successfully applied at various stages throughout the entire SDLC. One effective way to apply such knowledge is through the use of software security best practices such as the touchpoints. For example, rules are extremely useful for static analysis and code inspection activities.
Software security best practices and their associated knowledge catalogs can be applied regardless of the base software process being followed. Software development processes as diverse as the waterfall model, RUP, XP, Agile, spiral development, and CMMi (and any number of other processes) involve the creation of a common set of software artifacts (the most common artifact being code). Figure 11-2 shows an enhanced version of the touchpoints diagram that serves as the backbone of this book. In the figure, I identify those activities and artifacts most clearly impacted by the knowledge catalogs described here.
Figure 11-2. Mapping of software security knowledge catalogs to various software artifacts and software security best practices (the touchpoints described in this book).
The box Two Example Catalog Entries: A Principle and a Rule (see page 270) and the preceding Table 11-1 provide an overview of each of the knowledge catalogs. Principles, given their philosophical level of abstraction, bring significant value to early-lifecycle activities including the definition of security requirements, performance of software architecture risk analysis, and design reviews. Rules, given their tactical, specific, syntactic nature, are primarily applicable during implementation of code review and are particularly well suited for inclusion in a static analysis tool. This opportunity for automation means that rules have an implicit requirement for encapsulation in a deterministic definition language so that they can be consumed by automated code scanning software.
As you can see, this set of software security knowledge catalogs offers an excellent foundation for integrating security knowledge into the full SDLC.
The Department of Homeland Security Build Security In Portal[4]
The U.S. Department of Homeland Security is developing a software security portal (along with the Carnegie Mellon Software Engineering Institute and Cigital). This portal aims to provide a common, accessible, well-organized set of information for practitioners wishing to practice software security. The portal effort is expressly aimed at the problem of encapsulating, expanding, and spreading software security knowledge.
Like this book, the Build Security In (BSI) Software Assurance Initiative seeks to alter the way that software is developed by building security in from the start so that it’s less vulnerable to attack. BSI is a project of the Strategic Initiatives Branch of the Department of Homeland Security’s National Cyber Security Division (NCSD). NCSD sponsors development and collection of software assurance and software security information that will help software developers and architects create secure systems.
As part of the initiative, a BSI content catalog will be available on the US-CERT Web site <http://buildsecurityin.us-cert.gov/portal/>. This portal is intended for software developers and software development organizations who want information and practical guidance on how to produce secure and reliable software. The catalog is based on the principle that software security is fundamentally a software engineering problem that we must address systematically throughout the SDLC. The catalog contains and links to a broad range of information about best practices, tools, and knowledge.
Figure 11-3 identifies aspects of software assurance currently covered in the catalog. Material is divided into three major categories: best practices, tools, and foundational knowledge. This is an alternative way of organizing software security content with reference to artifacts.
Figure 11-3. The organizing concept for the BSI portal. The alignment of this view shows not only best practices (as Figure 11-2 does) but also knowledge and tools.
The categorization is the result of merging an earlier collaboration framework with ideas presented in the lifecycle touchpoints diagram that serves as the skeleton of this book. The National Cyber Security Taskforce’s report also identified additional practices to produce secure software—see <http://www.cyberpartnership.org/init-soft.html>. The BSI portal will supplement the taskforce’s practices with process models and references to appropriate tools, measurement, and other resources.
Although the team creating the portal won’t achieve complete content coverage immediately, DHS has launched the portal with some content in each area shown in Figure 11-3. The BSI team will use feedback received on this content (as well as input from industry) to prioritize further work on the catalog.
The portal includes several types of information, categorized for efficient search and utility as follows.
Best practices: A significant portion of the BSI effort is devoted to best practices that can provide the biggest return considering the current best thinking, available technology, and industry practice. This list will grow as more resources become available, more practices are proven, changes occur in the industry environment, and technology progresses. This book covers a number of critical best practices in some detail.
Knowledge: Software defects with security ramifications—including implementation bugs such as buffer overflows and design flaws such as inconsistent error handling—promise to be with us for years. Recurring patterns of software defects leading to vulnerabilities have been identified by long-time software security practitioners, and the BSI team is documenting detailed instructions on how to produce software without these defects. This work shows up in Figure 11-3 as “Guidelines” and “Coding rules.”
The BSI team has also identified principles that provide high-level direction for avoiding security problems in design, such as the principle of least privilege and the principle of compartmentalization. The BSI team is collaborating with the National Institute of Standards and Technology (NIST), the International Organization for Standardization (ISO), and the Institute of Electrical and Electronics Engineers (IEEE) on standards activities focused on developing safe and secure subsets of languages and software assurance style guides.
Tools: The BSI portal includes information about which tools developers and security analysts can use to detect and/or remove common vulnerabilities. Of particular interest are static analysis tools that help developers look for common security-critical problems in source code. The best current commercial tools support languages like Java, CLR, C++, C, and PHP.
Business case: Even with extensive technical content, a business case is required to convince industry to adopt secure software development best practices and educate consumers about the need for software assurance. Therefore, each documented best practice addresses the business case for use of that practice. In addition, we’ve included an overall business case framework.
Dynamic navigation: The extent to which users will find the content accessible as well as useful will determine how this portal will impact real-world development practices and, thus, overall systems security. The BSI team is making the content approachable in several different ways. For example, a software engineer might use the catalog to determine applicable security guidelines, an architect might use security principles to determine how to design an n-tier application in a secure fashion, and a development team leader might use the information to justify software assurance techniques to management by building a business case. Because the repository will be structured and designed to evolve as well as support usage by a variety of user types, it includes a dynamic navigation interface.
Once practical guidance and reference materials are available for the day-to-day work most development organizations do, the BSI team plans to identify and organize content for practical guidance and reference materials for enterprise-level security concerns.
Although the portal is currently in a nascent stage, the BSI team welcomes feedback on this effort. Information on providing feedback can be found on the portal itself; community involvement and use is crucial to its success.
Efforts to identify and define knowledge constructs for software security are in their infancy. My hope is that a wider population of thought leaders and key practitioners of software security will help to refine and validate this knowledge architecture in an effort to build consensus and move toward standardization. Such discussion and collaboration are critical to the success of software security as a unified practice. As work continues to gain consensus, my colleagues and I will continue to collect real-world examples of content to build out the breadth and depth of catalogs. We will also work to identify further opportunities for directly applying these catalogs in the SDLC.
There is really no better time to get into software security than now. The field is beginning to explode, mostly due to incredible commercial demand. Turns out that we’ve built boatloads of pretty bad software over the years, and now that security is being taken more seriously, there’s one heck of a cleanup job to do. That’s right, we can’t solve the problem in “look ahead” mode only. We need to spend some time fixing what we’ve already built. The cool thing about the touchpoints is that many can be applied just as well to existing software as to new projects. For example, performing an architectural risk analysis on an existing system is well within the realm of possibility.
Getting started in software security is easier than ever. Now there’s an entire shelf full of software security books (see Chapter 13), best practices like the touchpoints have been identified, and organizations are looking to build capability. Knowledge managers are creating schemata and taxonomies of software security knowledge, making it much easier than it was just a few short years ago to get started. And the tools don’t suck anymore.
If you are a software person interested in security, consider becoming a software security person. We need you!
[1] Parts of this chapter appeared in original form in IEEE Security & Privacy magazine as two articles, one coauthored with Sean Barnum [Barnum and McGraw 2005] and one with Nancy Mead [Mead and McGraw 2005].
[2] If you want to experience firsthand why reading rules is tedious, check out Appendix B.
[3] Mitre’s CVE <http://www.cve.mitre.org/>, the CERIAS database <https://cirdb.cerias.purdue.edu/coopvdb/public/>, and CERT’s alert data <http://www.cert.org> are three popular collections.
[4] Known in government acronym speak as the DHS BSI PRTL.