There is an old saying in the consulting business: "If you do not document it, it did not happen." Of course, the insinuation here is that because it did not happen, you cannot bill for it. Whether you are working as a consultant or as a full-time employee, failing to report the findings of your security assessment, in a format and style that results in improvements to security, will render your work academic. Too frequently, good work is dismissed because findings are reported in an unprofessional manner, without appropriate focus on their impact on core business operations, or without adequate justification. If you are running a vulnerability scanning project, penetration test, or IT security audit, this chapter provides you with a framework for reporting your findings that will result in improved security for your organization—and maybe even a raise for you!
Before learning about the reporting framework, you should be aware of some global guidelines that pertain to style, which will help you shape your report. After you complete your final report at the conclusion of your security assessment, be sure it meets these criteria:
Concise and professional
Technically accurate
Objective
Measurable
Because security assessments inherently contain recommendations that IT management might not want to hear, namely that necessary security improvements will be costly or will delay the completion of a project, IT management might use any excuse to diminish the accuracy or importance of your work. Consequently, one of the primary guidelines for reporting your findings is to avoid common writing mistakes that will distract attention from your recommendations. Your report should be complete yet concise. If you are overly detailed or wordy, there is a good chance that no one will read your report, so keep in mind what your readers need to know and what questions they are likely to ask. If those questions guide the development of your report, your recommendations will be much better received. Numerous mistakes in grammar, punctuation, and spelling are distracting. At a minimum, build time into your project to proofread your report; ideally, a couple of other people review your writing before you generate your final report. Your report should contain the following parts:
Cover sheet. The cover sheet should contain the title of your report, names of the principle authors, data, and a brief abstract of the project.
Table of Contents. If your report is over four pages long, which is almost a certainty, include a Table of Contents for reference.
Executive summary. The execute summary provides your readers with an overall summary of the results of the project in no more than one page. Pay special attention to writing this part well—this might be the only page management reads.
Summary of work. In this section, briefly discuss the scope of the project, its goals, and the methodology you used to meet the goals.
Detailed findings. In this section, document your findings in detail. What you should include in this section is discussed in depth later in this chapter.
Reference citations. Use a well-known bibliography standard, such as the Chicago Manual of Style, to document sources that you have cited within the report.
This should go without saying, but here it is anyway: because security assessments judge the work of others, the assessment team’s findings will be held to a higher standard, and thus should be technically accurate. Nothing will discredit you and your work more than technical inaccuracies in your report. More importantly, there might not be anyone to correct your technical mistakes, which could lead to the creation of new security vulnerabilities or could leave existing vulnerabilities. Verify your assertions when you have any doubt about them. To bolster the credibility of your findings, consider adding a citation in your report referring readers to information about a vulnerability or a countermeasure for a vulnerability.
Security professionals are world-renowned for being paranoid. Even though paranoia is sometimes a job requirement, it causes security professionals to be viewed as alarmists. When you construct the report of your findings, be careful to avoid statements that are inflammatory, unsupported by the evidence, speculative, or overly frightening. The best method for doing this is to focus on the solution to the problem rather than on the problems you find. This will help focus your readers’ attention on the future (which you hope is more positive than the past). For example, you would not want to start off your penetration testing results with the following statement:
"The Web application is so porous because of poor development practices that portions will likely need to be rewritten from scratch."
This statement is inflammatory and speculative. Compare it with the following statement:
"In the Web application, we found several instances of SQL injection that were easily exploitable. By rewriting these portions of the application to use parameterized stored procedures, these vulnerabilities will be eliminated."
The second example is clearly much less alarmist and much more helpful. In fact, the second statement is reassuring.
For any security measure that you recommend in your findings, whether corrective or preventive, be sure to include information about how it can be verified or measured. This will not only help provide a basis for future security assessments, but also provide a measure of success for your assessment program, which you can present to IT management. If possible, tie these measurements to cost savings or other direct benefits to the business, such as providing assurance for regulatory compliance. Doing so will help justify the costs of continuing security assessment and improvement. The bottom line is that organizations value what they measure and measure what they value. Thus, the more you can measure security improvements resulting from your security assessment, the more the organization will value the service you are providing.
Odds are that unless the scope of your security assessment is very small and only a few areas need improvement, you will have a fairly large amount of information to report. To make the information useful to IT management and administrators, your findings must be easy to read and understand. That said, you probably won’t have all the time that you want to prepare your findings. To meet these conflicting goals, stick to a reporting framework that focuses on key elements, rather than on the minutia. For any area that your security assessment finds deficient, you must answer these four questions and include these answers in your report:
What risk does the vulnerability present?
What should be done to mitigate the vulnerability?
Where should the mitigation be done?
Who should be responsible for implementing the mitigations?
When defining the vulnerability, you are really answering these questions:
What is the source of the vulnerability?
What is the potential impact of the vulnerability?
What is the likelihood of the vulnerability being exploited?
Regarding the source of the vulnerability, be as precise as possible and provide an external reference for your readers if they are not familiar with the vulnerability. Pay particular attention to ensuring that you are identifying the source of the vulnerability and not the symptom of it. Failure to identify the source could allow the cause to persist. For example, numerous instances of SQL injection and cross-site scripting can be indicative of untrained developers. Although removing the SQL injection and cross-site scripting will improve the security of the affected website, the developers who wrote the code might continue to introduce these problems in new code.
When you describe the impact of the vulnerability, be direct and honest while creating a mental image for your readers. For penetration testing in particular, you can illustrate and be specific about the impact because you were able to see it first-hand during the assessment. For vulnerability scanning or IT security audits, you might need to be a little creative to create a scenario that resonates with your readers. In addition to describing the potential impact of the vulnerability, you might want to assign a numerical value for comparison. For example, you could use a 10-point scale that has four levels, as shown in Table 6-1.
Table 6-1. Ranking of Potential Impact of Vulnerabilities
Level | Numeric value | Potential impact |
|---|---|---|
Critical | 9–10 | Vulnerabilities ranked as critical have impacts that are well beyond the scope of the organization and its information assets, including consequences such as: |
Loss of life or physical injury | ||
Compromise of critical infrastructure or financial infrastructure systems | ||
Damage to the brand of the organization | ||
Lawsuits incurred from regulatory agencies | ||
High | 6–8 | Vulnerabilities ranked as high have impact on core business operations, including: |
Remote compromise of servers | ||
Escalation of privilege to multi-system administrative capabilities (that is, the domain administrator) | ||
Compromise of high-value information assets, like customer databases | ||
Denial of service to mission-critical IT operations | ||
3–5 | Vulnerabilities ranked as medium have localized impact on IT or secondary business operations, including: | |
Loss of productivity, such as that seen with many viruses | ||
System instability | ||
Local escalation of privilege | ||
Low | 1–2 | Vulnerabilities ranked as low have minimal direct impact on information assets and include consequences such as: |
Non-critical information disclosure | ||
Minor disruption of service |
In addition to stating the potential impact, you should also state the likelihood of a vulnerability being exploited. Because predicting the probability that a given vulnerability will occur can be difficult, you might find it helpful to think of the likelihood as a combination of the following:
Access
Difficulty
Value of the asset to the attacker
Just as you can rank the potential impact of the vulnerability, you can rank the likelihood an attacker will attempt to exploit a vulnerability. You do this by giving three elements—access, difficulty, and value of the asset to the attacker—a score from 0 through 3. Then add those scores and add 1 to create a maximum value of 10.
The more access attackers have to an information asset, the greater the compromise will probably be. This level of access is often referred to as the attack surface. Consider the degree of access to the information asset both physically and over the network and the intermediate security dependencies. For example, attackers will have greater access to a Web server exposed to the Internet than one exposed only to the intranet. Similarly, if exploiting the vulnerability is predicated on having administrator access, access is already very limited.
The more difficult it is to construct or carry out an exploit, the less likely that attack will occur. Frequently, the difficulty of exploiting a vulnerability changes over time. For example, before July 16, 2003, the difficulty of exploiting the buffer overrun in DCOM over RPC required both the skill to find the buffer overrun and the skill to write the exploit code. On July 16, the security bulletin MS 03-026 was published. This decreased the difficulty of finding the buffer overrun. By July 28, source code for exploiting the product vulnerability was widely distributed on the Internet. At this point, exploiting the vulnerability took neither the skill to locate the buffer overrun nor the skill to craft the exploit.
Though not always a perfect indicator, the value of the asset to the attacker is part of the likelihood equation. The tricky issue here is determining what assets attackers find valuable. For example, the Microsoft Web properties Microsoft.com, MSN.com, and Hotmail.com all have value to attackers in the attention that they would bring if compromised. Similarly, professional attackers are more likely to attack financial institutions or police networks than small businesses, because these networks contain more valuable information.
After you estimate both the potential impact and the likelihood of exploitation, you can multiply the two values to get a relative indicator of the amount of risk the vulnerability presents to your organization. The complete equation looks like:
Relative risk = Potential impact × (Access + Difficulty + Value to attacker + 1)
Rating the overall risk will help you prioritize which weaknesses need to be addressed. Using this formula, the maximum relative risk will be 100.
Presenting a vulnerability in your findings without documenting how the vulnerability could be managed is only half of your security assessment job. The other half is presenting potential solutions, mitigations, or other suggestions for reducing or eliminating the vulnerability. Be careful not to present an all-or-nothing solution when other options are possible, especially when the ideal solution is costly. In these situations, present three options and their costs (direct and indirect). The first solution describes the ideal, the second solution describes a workable and acceptable scenario, and the third solution offers a minimal situation. Presenting your guidance in this manner helps accomplish two goals. First, you shift the burden of making security decisions to management, where it belongs. Second, you create a psychological trap. For the most part, when people are faced with three options and have limited resources, they have a strong tendency to choose the middle option because of cost, because of their own perception of risk. This tendency is especially powerful when their third option is to accept the risk that the existing situation presents.
For example, suppose that in your security evaluation, you find several instances in which weak authentication protocols are being used to validate accounts for accessing a customer database through the ASP Web application. Instead of issuing a single remediation, you could give these three options:
Implement certificates to all users of the customer database and require certificate-based authentication on the front-end website in addition to the forms-based authentication on the website. This solution will require the design and implementation of a PKI and Active Directory. Additionally, all client operating systems must run Microsoft Windows 2000 or later.
Migrate the accounts database to Active Directory and implement basic authentication over SSL on the website. This solution will require the design and implementation of Active Directory.
Continue to use the current custom authentication protocol, which is highly susceptible to spoofing or man-in-the-middle attacks.
Given these options, even in abbreviated form, IT managers will be highly inclined to choose the second option. Although you might not be able to use this model for all situations because some security measures will have no alternatives, more often than not you will.
Often, eliminating a system vulnerability in one place just shifts the focus of the attackers to other, weaker parts of the system. Consequently, when presenting your findings, take time to think through and document all places in which changes should take place—especially when performing IT security audits. One model you can use analyzes your recommendations for incompleteness and is shown in Table 6-2. You can create a spreadsheet with these areas as columns and describe how your solution will affect each area.
Table 6-2. Areas of a Complete Security Recommendation
Key question | |
|---|---|
Security policy | What changes in the organization’s security policy will be required, either directly or indirectly? |
Process and procedures | What processes and procedures will need to be created or modified to meet the recommendations? |
Technology | What technology will be used in the solution? |
Implementation | How should the recommendations, technical or nontechnical, be implemented, and how can users or administrators comply with the recommendations? |
Documentation | What should be added, modified, or removed from network diagrams or documentation as a result of the changes? |
Operations | How will the daily maintenance and management of the IT systems change? Is training required? |
Your report might not need to answer all the questions in Table 6-2, but at a minimum, think about the questions to help you craft a more complete solution to the vulnerabilities you discover. By working through these issues in your mind, you will be much better prepared to discuss your recommendations with IT management.
You can’t always assign the responsibility for implementing approved recommendations, especially when you are a consultant to the organization whose security you are assessing. Ideally in your report, you would include the individual responsible for carrying out the recommendations. If nothing else, your report should indicate the general parties that should implement the recommendations to help IT management assign responsibility for making the changes.