CHAPTER 9
Managing a Penetration Test
In this chapter, we discuss managing a penetration test. We cover the following topics:
• Planning a penetration test
• Structuring a penetration testing agreement
• Execution of a penetration test
• Information sharing during a penetration test
• Reporting the results of a penetration test
When it comes to penetration testing, the old adage is true: plan your work, then work your plan.
Planning a Penetration Test
When planning a penetration test, you will want to take into consideration the type, scope, locations, organization, methodology, and phases of the test.
Types of Penetration Tests
There are basically three types of penetration testing: white box, black box, and gray box.
White Box Testing
White box testing is when the testing team has access to network diagrams, asset records, and other useful information. This method is used when time is of the essence and when budgets are tight and the number of authorized hours is limited. This type of testing is the least realistic, in terms of what an attacker may do.
Black Box Testing
Black box testing is when there is absolutely no information given to the penetration testing team. In fact, using this method of testing, the penetration testing team may only be given the company name. Other times, they may be given an IP range and other parameters to limit the potential for collateral damage. This type of testing most accurately represents what an attacker may do and is the most realistic.
Gray Box Testing
Gray box testing is, you guessed it, somewhere in between white box testing and black box testing. This is the best form of penetration testing where the penetration testing team is given limited information and only as required. So, as they work their way from the outside in, more access to information is granted to speed the process up. This method of testing maximizes realism while remaining budget friendly.
Scope of a Penetration Test
Scope is probably the most important issue when planning a penetration test. The test may vary greatly depending on whether the client wants all of their systems covered or only a portion of them. It is important to get a feel for the types of systems within scope to properly price out the effort. The following is a list of good questions to ask the client (particularly in a white box testing scenario):
• What is the number of network devices that are in scope?
• What types of network devices are in scope?
• What are the known operating systems that are in scope?
• What are the known websites that are in scope?
• What is the length of the evaluation?
• What locations are in scope?
Locations of the Penetration Test
Determining the locations in scope is critical to establishing the amount of travel and the level of effort involved for physical security testing, wireless war driving, and social engineering attacks. In some situations, it will not be practical to evaluate all sites, but you need to target the key locations. For example, where are the data centers and the bulk of users located?
Organization of the Penetration Testing Team
The organization of the penetration testing team varies from job to job, but the following key positions should be filled (one person may fill more than one position):
• Team leader
• Physical security expert
• Social engineering expert
• Wireless security expert
• Network security expert
• Operating System expert
Methodologies and Standards
There are several well-known penetration testing methodologies and standards.
OWASP
The Open Web Application Security Project (OWASP) has developed a widely used set of standards, resources, training material, and the famous OWASP Top 10 list, which provides the top ten web vulnerabilities and the methods to detect and prevent them.
OSSTMM
The Open Source Security Testing Methodology Manual (OSSTMM) is a widely used methodology that covers all aspects of performing an assessment. The purpose of the OSSTMM is to develop a standard that, if followed, will ensure a baseline of test to perform, regardless of customer environment or test provider. This standard is open and free to the public, as the name implies, but the latest version requires a fee for download.
ISSAF
The Information Systems Security Assessment Framework (ISSAF) is a more recent set of standards for penetration testing. The ISSAF is broken into domains and offers specific evaluation and testing criteria for each domain. The purpose of the ISSAF is to provide real-life examples and feedback from the field.
Phases of the Penetration Test
It is helpful to break a penetration test into phases. For example, one way to do this is to have a three-phase operation:
• I: External
• II: Internal
• III: Quality Assurance (QA) and Reporting
Further, each of the phases may be broken down into subphases; for example:
• I.a: Footprinting
• I.b: Social Engineering
• I.c: Port Scanning
• II.a: Test the internal security capability
• And so on.
The phases should work from the outside to the inside of an organization, as shown in Figure 9-1.
Figure 9-1 Three-phase penetration testing plan
Notice in Figure 9-1 phase II.a, Test Security Response. The purpose of this phase is to test the client’s security operations team. If done properly and coordinated with the fewest amount of people possible, this phase is quite effective in determining the security posture of an organization. For example, it helps to determine whether or not the security team responds to network scans or deliberate attacks on the network. This phase can be done onsite or offsite with a VPN connection. This phase is normally short, and once the results are noted, the assessment moves on to the next phase, with or without the cooperation of the security operations team (depending on the type of assessment performed).
Testing Plan for a Penetration Test
It is helpful to capture the plan and assignments on a spreadsheet. For example:
A spreadsheet like this allows you to properly load balance the team and ensure that all elements of the phases are properly scheduled.
References
Penetration test http://en.wikipedia.org/wiki/Penetration_test
Good list of tasks www.vulnerabilityassessment.co.uk/Penetration%20Test.html
Structuring a Penetration Testing Agreement
When performing penetration tests, the signed agreements you have in place may be your best friend or worst enemy. The following documents apply.
Statement of Work
Most organizations use a Statement of Work (SOW) when contracting outside work. The format of the SOW is not as important as its content. Normally, the contractor (in this case, the penetration tester) prepares the SOW and presents it to the client as part of the proposal. If the client accepts, the client issues a purchase order or task order on the existing contract. There are some things you want to ensure you have in the SOW:
• Purpose of the assessment
• Type of assessment
• Scope of effort
• Limitations and restrictions
• Any systems explicitly out of scope
• Time constraints of the assessment
• Preliminary schedule
• Communication strategy
• Incident handling and response procedures
• Description of the task to be performed
• Deliverables
• Sensitive data handling procedures
• Required manpower
• Budget (to include expenses)
• Payment terms
• Points of contact for emergencies
Get-Out-of-Jail-Free Letter
Whenever possible, have the client give you a “get-out-of-jail-free letter.” The letter should say something like
To whom it may concern,
Although this person looks like they are up to no good, they are actually part of a security assessment, authorized by The Director of Security...
Please direct any questions to...
A letter of this sort is particularly useful when crawling around dumpsters in the middle of the night.
References
NIST Technical Guide to Information Security Testing and Assessment (800-115; replaces 800-42) csrc.nist.gov/publications/nistpubs/800-115/SP800-115.pdf
OSSTMM www.isecom.org/osstmm/
Execution of a Penetration Test
Okay, now that we have all the planning and paperwork in place, it is time to start slinging packets...well, almost. First, let’s get some things straight with the client.
Kickoff Meeting
Unless a black box test is called for, it is important to schedule and attend a kickoff meeting, prior to engaging with the client. This is your opportunity not only to confirm your understanding of the client’s needs and requirements but also to get off on the right foot with the client.
It is helpful to remind the client of the purpose of the penetration test: to find as many problems in the allotted time as possible and make recommendations to fix them before the bad guys find them. This point cannot be overstated. It should be followed with an explanation that this is not a cat-and-mouse game with the system administrators and the security operations team. The worst thing that can happen is for a system administrator to notice something strange in the middle of the night and start taking actions to shut down the team. Although the system administrator should be commended for their observation and desire to protect the systems, this is actually counterproductive to the penetration test, which they are paying good money for.
The point is that, due to the time and money constraints of the assessment, the testing team will often take risks and move faster than an actual adversary. Again, the purpose is to find as many problems as possible. If there are 100 problems to be found, the client should desire that all of them be found. This will not happen if the team gets bogged down, hiding from the company employees.
NOTE
As previously mentioned, there may be a small phase of the penetration test during which secrecy is used to test the internal security response of the client. This is most effective when done at the beginning of the test. After that brief phase, the testing team should move as fast as possible to cover as much ground as possible.
Access During the Penetration Test
During the planning phase, you should develop a list of resources required from the client. As soon as possible after the kickoff meeting, you should receive those resources from the client. For example, you may require a conference room that has adequate room for the entire testing team and its equipment and that may be locked in the evenings with the equipment kept in place. Further, you may require network access. You might request two network jacks, one for the internal network, and the other for Internet access and research. You may need to obtain identification credentials to access the facilities. The team leader should work with the client point of contact to gain access as required.
Managing Expectations
Throughout the penetration test, there will be a rollercoaster of emotions (for both the penetration testing team and the client). If the lights flicker or a breaker blows in the data center, the penetration testing team will be blamed. It is imperative that the team leader remain in constant communication with the client point of contact and manage expectations. Keep in mind this axiom: first impressions are often wrong. As the testing team discovers potential vulnerabilities, be careful about what is disclosed to the client, because it may be wrong. Remember to under-promise and overachieve.
Managing Problems
From time to time, problems will arise during the test. The team may accidentally cause an issue, or something outside the team’s control may interfere with the assessment. At such times, the team leader must take control of the situation and work with the client point of contact to resolve the issue. There is another principle to keep in mind here: bad news does not get better with time. If the team broke something, it is better to disclose it quickly and work to not let it happen again.
Steady Is Fast
There is an old saying, “steady is fast.” It certainly is true in penetration testing. When performing many tasks simultaneously, it will seem at times like you are stuck in quicksand. In those moments, keep busy, steadily grinding through to completion. Try to avoid rushing to catch up; you will make mistakes and have to redo things.
External and Internal Coordination
Be sure to obtain client points of contact for questions you may have. For example, after a couple of days, it may be helpful to have the number of the network or firewall administrator on speed dial. During off hours, if the client point of contact has gone home, sending an e-mail or SMS message to them occasionally will go a long way toward keeping them informed of progress. On the other hand, coordination within the team is critical to avoid redundancy and to ensure that the team doesn’t miss something critical. Results should be shared across the team, in real time.
Information Sharing During a Penetration Test
Information sharing is the key to success when executing a penetration test. This is especially true when working with teams that are geographically dispersed. The Dradis Server is the best way to collect and provide information sharing during a penetration test. In fact, it was designed for that purpose.
Dradis Server
The Dradis framework is an open source system for information sharing. It is particularly well suited for managing a penetration testing team. You can keep your team informed and in sync by using Dradis for all plans, findings, notes, and attachments. Dradis has the ability to import from other tools, like
• Nmap
• Nessus
• Nikto
• Burp Scanner
NOTE
The Dradis framework runs on Windows, Linux, Mac OS X, and other platforms. For this chapter, we will focus on the Windows version.
Installing Dradis
You can download the Dradis server from the Dradis website, http://dradisframework.org. After you download it onto Windows, execute the installation package, which will guide you through the installation.
NOTE
The Dradis installer will install all of the prerequisites needed, including Ruby and SQLite3.
Starting Dradis
You start the Dradis framework from the Windows Start menu.
It takes a few moments for the Ruby Rails server to initialize. When the startup screen looks like the following, you are ready to use the server.
Next, browse to
After you get past the warnings concerning the invalid SSL certificate, you will be presented with a welcome screen, which contains useful information.
User Accounts
Although there are no actual user accounts in Dradis, users must provide a username when they log in, to track their efforts. A common password needs to be established upon the first use.
Clicking the “back to the app” link at the top of the screen takes you to the Server Password screen.
The common password is shared by all of the team members when logging in.
NOTE
Yes, it is against best practice to share passwords.
Interface
The user interface is fashioned after an e-mail client. There are folders on the left and notes on the right, with details below each note.
The Dradis framework comes empty. However, in a few minutes’ time, you may add nodes and subnodes to include anything you like. For example, as just shown, you may add a node for your favorite methodology and a node for the vulnerabilities that you found. This allows you to use the system as a kind of checklist for your team as they work through the methodology. You may add notes for each node and upload attachments to record artifacts and evidence of the findings.
Export/Upload Plug-ins
A very important capability of Dradis is the ability to export and import. You may export reports in Word and HTML format. You may also export the entire database project or just the template (without notes or attachments).
This allows you to pre-populate the framework on subsequent assessments with your favorite template.
Import Plug-ins
There are several import plug-ins available to parse and import external data:
• WikiMedia wiki Used to import data from your own wiki
• Vulnerability Database Used to import data from your own vulnerability database
• OSVDB Used to import data from the Open Source Vulnerability Database
In order to use the OSVDB import plug-in, you need to first register at the OSVDB website and obtain an API key. Next, you find and edit the osvdb_import.yml file in the following folder:
C:Users<username goes here>AppDataRoamingdradis-2.5serverconfig>
Inside that file, edit the API key line and place your key there:
# Please register an account in the OSVDB site to get your API key. Steps:
# 1. Create the account: http://osvdb.org/account/signup
# 2. Find your key in http://osvdb.org/api
API_key: <your_API_key>
Save the file and restart your Dradis server. Now, you should be able to import data from the OSVDB site. At the bottom of the Dradis screen, click the Import tab. Select the External Source of OSVDB. Select the Filter as General Search. Provide a Search For string and press ENTER. It will take the OSVDB database a few seconds to return the results of the query. At this point, you can right-click the result you are interested in and import it.
Now, back on the Notes tab, you may modify the newly imported data as needed.
Team Updates
The real magic of Dradis occurs when multiple users enter data at the same time. The data is synchronized on the server, and users are prompted to refresh their screens to get the latest data. Access may be granted to the client, enabling them to keep abreast of the current status at all times. Later, when the assessment is done, a copy of the framework database may be left with the client as part of the report. Goodbye, spreadsheets!
References
Dradis http://dradisframework.org
OSVDB signup http://osvdb.org/account/signup
Reporting the Results of a Penetration Test
What good is a penetration test if the client cannot decipher the results? Although the reporting phase sometimes is seen as an afterthought, it is important to focus on this phase and produce a quality product for the client.
Format of the Report
The format of the report may vary, but the following items should be present:
• Table of contents
• Executive summary
• Methodology used
• Prioritized findings per business unit, group, department
• Finding
• Impact
• Recommendation
• Detailed records and screenshots in the appendix (back of report)
Presenting the findings in a prioritized manner is recommended. It is true that not all vulnerabilities are created equal. Some need to be fixed immediately, whereas others can wait. A good approach for prioritizing is to use the likelihood of remote administrative compromise. Critical findings may lead to remote administrative compromise (today) and should be fixed immediately. High findings are important but have some level of mitigation factor involved to reduce the risk to direct compromise. For example, perhaps the system is behind an internal firewall and only accessible from a particular network segment. High findings may need to be fixed within six months. Medium findings are less important and should be fixed within one year. Low findings are informational and may not be fixed at all. The recommended timelines may vary from tester to tester, but the priorities should be presented; otherwise, the client may be overwhelmed with the work to be performed and not do anything.
Presenting the findings grouped by business unit, group, or division is also recommended. This allows the report to be split up and handed to the relevant groups, keeping the sensitive information inside that group.
Out Brief of the Report
The out brief is a formal meeting to present a summary of the findings, trends, and recommendations for improvement. It is helpful to discover how the client is organized. Then, the out brief may be customized per business unit, group, or department. It may be helpful to deliver the findings to each group, separately. This reduces the natural tendency of defensiveness when issues are discussed among peers groups. If there is more than a week between the commencement of the penetration test and the actual out brief, a quick summary of critical findings, trends, and recommendations for improvement should be provided at the end of the assessment. This will allow the client to begin correcting issues prior to the formal out brief.