Planning and Scoping

Domain Objectives

•  1.1   Explain the importance of planning for an engagement.

•  1.2   Explain key legal concepts.

•  1.3   Explain the importance of scoping an engagement properly.

•  1.4   Explain the key aspects of compliance-based assessments.

Objective 1.1   Explain the importance of planning for an engagement

Without appropriate planning, engagements run the risk of failure due to inadequate buy-in, irrelevant results, breaches of contract, or even legal issues. To avoid these pitfalls, penetration testers need to do each of the following:

•   Gain an understanding of the target, including the people, technology, data, and political landscape that may influence testing

•   Determine rules that govern how testing can be conducted (rules of engagement)

•   Define what resources and information are required for testing and gather them

•   Understand requirements for communication, including stakeholders and the confidentiality, timeliness, and content of communication

•   Know how budget, remediation timelines, and impact analysis affect test delivery

•   Enumerate any limitations of testing or disclaimers that need to be considered by all parties when analyzing the testing results

This module will walk through each of these topics to highlight areas of critical importance for each of these considerations.

Understanding the Target Audience

In order to know what needs to be tested, how it needs to be tested, and what outcomes are expected as a result of testing, penetration testers need to come to an understanding with the client. This understanding should help the penetration tester grasp details about the testing environment, target, organization, and political landscape that are necessary to define the correct test during scoping. (Scoping is covered in more detail in Objective 1.3.)

Knowing what is most important to the target organization ensures that the right things are tested, in the right way, to meet organizational goals and objectives. To get this information, communications are key. Testers must understand the people who are participating in the process and understand what information those people expect and can provide.

Stakeholders may be responsible for

•   Defining the test

•   Performing the testing

•   Approving or financing the testing

•   Fixing things that are found or caused by the test

•   Overseeing the test, including contracts or communication

Typical penetration tests will use a mixture of stakeholders from some (or all) of the following groups. There may be cases where the organization wishes to limit who knows about the test or the results during testing. This will define the target audience for communications during testing. Designation of a single point of contact and a list of who is allowed to receive information about the engagement often addresses these concerns and reduces confusion during the test. Testing stakeholders and target audience members may be composed of combinations of these people:

•   Executive Management   Senior leaders with authority over budget, approval for testing for systems owned by the client, and who leads the organization and its strategy.

These personnel need to understand findings relevant to laws, rules, and regulations, as well as strategic analysis of the findings that is pertinent to inform decisions about the organization’s overall success. These personnel often appreciate a “bottom-line, up-front” communication approach that presents technical issues in concise terms that are accessible to all levels of expertise.

•   Security Staff   Management in this area may hold authority over budget and approval for testing parameters and may be responsible for directing actions required as a result of the test findings.

Security personnel need details about the weaknesses identified, their impact and relative severity, and may require details about timelines and specific targets during testing activity.

•   IT Staff   Technical resources who are responsible for systems implementation and configuration. Often responsible for taking action as required by test findings. May include application developers, application administrators, database administrators, server administrators, workstation administrators, network administrators or engineers, and others depending on the kinds of systems being tested and the types of testing being conducted.

These personnel need details about specific systems or settings affected by testing and may need timelines of testing activity. However, they may care more about how to change a system in order to prevent an attack than about understanding how an attack works.

•   Contracting and Legal Staff   Ensure that contractual commitments are agreeable by both parties and that the agreements are upheld. May be consulted to review contracts and advise about legal issues as part of test planning. As an example, consultants testing certain federal facilities may require special credentials or approvals, which may require review by a legal representative.

Generally, legal or contracting staff will approve the text of the agreement before executive management signs to approve the terms.

•   Penetration Testers   Penetration testers may be internal or external to the organization being tested. Testers are expected to follow all of the agreed-upon terms of testing, to execute the test, and to communicate about the results.

These individuals are expected to be experts in their field with a firm grasp of technical detail and the ability to explain it clearly to any level of technical expertise.

•   Third Parties   Third-party providers, such as cloud service providers, may need to provide additional testing authorization, depending on the systems being tested and the kinds of testing being done.

Often, third parties have set processes to request testing approval.

Rules of Engagement

Testers and stakeholders must agree on a set of rules that defines how testing may and may not be done. These rules cover things like

•   When testing may occur within the start and end dates that have been agreed upon—for example, during what days or hours testing may occur.

•   Data handling expectations, such as whether testers should access certain types of information during testing and how such access is permitted.

•   Where is the tester or testing appliance deployed during the test (onsite, offsite, and where, specifically)?

•   Who is allowed to know that testing is occurring within the target organization?

•   What (generally) is tested, and what is not? More details about this are often described in scoping documentation.

•   What testing techniques are allowed, and what techniques are forbidden? For example, is brute-forcing of passwords allowed and, if so, how many attempts may be made per account? Is social engineering allowed, or only systems/network testing?

Communication

Testers should define communication requirements with stakeholders as part of test planning. A communication plan defines the chain of command for decision making during test planning and other considerations such as

•   What information should be shared, with whom, and how should it be transmitted?

•   Who are the appropriate points of contact for escalations?

•   What is the appropriate method for communication?

•   What are the shared contact details (phone, e-mail) for communication?

•   What triggers official communications?

•   How frequent should communication be?

Here are some examples of communication that may occur during a penetration test:

•   Escalation of issues

•   Testing status reports between penetration testers

•   Testing status reports between penetration testers and target stakeholders

•   Specific milestones, such as goal attainment, daily start/end of testing, or phase of testing updates

•   Possible adjustments to the test parameters

Testers can escalate decisions about changes to testing that might introduce additional impact within the chain of command. These issues can be discussed with a penetration test manager or a designated point of contact within the target organization so that a decision can be made. This shared decision-making process protects the tester from negative outcomes of these decisions. Here are a few examples of issues that may need to be escalated during a penetration test:

•   The tester identifies evidence of prior system compromise during testing.

•   The tester causes an unexpected outage during testing.

•   The tester finds a possible vulnerability that needs to be confirmed using techniques with the potential of additional impacts that require additional discussion with the stakeholders.

•   An incident occurs within the target organization, and testing must be paused or halted while a response takes place.

•   A serious vulnerability is confirmed, and the potential impact suggests that the details should be disclosed to the target audience before the final report in order to expedite remediation.

Resources and Requirements

Stakeholders and penetration testers work together to define what resources are available to the tester and what requirements are necessary to ensure the test is successful. This includes defining the minimum information the tester needs to conduct testing, steps that need to be taken before testing can commence, and any equipment/implementation a tester needs to conduct testing. (See the section “Support Resources” for additional information.)

Confidentiality of Findings

Penetration testers are likely to have access to sensitive information about the target organization as a result of testing. This may be knowledge about significant system weakness, or even access to data that is otherwise confidential to the organization being tested. To protect the organization and its information, target stakeholders typically insist that all information and findings resulting from a penetration test are confidential and should be shared only with the appropriate target audience members. (Read more about confidentiality requirements in the “Contracts” section of Objective 1.2, where NDAs are discussed.)

Known vs. Unknown

Test plans are often somewhat fluid due to the need to address contingency planning. However, it is difficult—even impossible—to predict every contingency. When issues arise during testing that are unexpected, the target organization or the tester may need to revise requirements to compensate. Here are examples of known and unknown requirements:

•   Known requirement   For a physical penetration test, the target organization wants the tester to evaluate the integrity of locked doors allowing ingress into a target facility. The target organization provides a diagram of all doors and a list of the lock types used. The doors all use physical locks. The tester would know that tools for bypassing physical locks are a requirement for testing.

•   Unknown requirement   During the same penetration test, the tester discovers an undocumented door that relies on a magnetic lock rather than a physical lock. Upon clarification with the target point of contact, the door should be tested. To test this door, additional tools are required beyond what was determined during examination of the original documentation. This is an unknown requirement that may require the tester to adapt testing methodology and tooling.

Budget

The budget is the amount of money an organization can spend for testing, including all costs related to testing. This can determine whether the test is a remote test or an onsite test, as budget must be allocated for travel of testing staff. This can also determine how many people or hours are able to be allocated for the engagement. That, in turn, may mean a representative subset of systems should be scoped for testing, rather than the entirety of all systems.

However, these decisions are typically made between the testing organization and the target organization in order to ensure that the appropriate quality of service can be achieved and that the target organization’s goals are met within these budgets. In some cases, the scope and method of testing cannot be compromised for the sake of budget. One example is compliance-based testing, which may be governed by specific testing requirements. (Read more about compliance-based assessment requirements in Objective 1.4.)

Impact Analysis and Remediation Timelines

Understanding the potential impact of testing methodologies on targets helps target organizations understand and mitigate any risk from testing by supplying limitations on testing scope and methods, time frames, and planning for the contingencies of impact. Not all impacts can be predicted, so testers must discuss potential impacts as well as expected outcomes in order to allow the target audience to make appropriate determinations that will enable a test that meets the organization’s objectives. Impact analysis examples that may be discussed during planning include

•   Brute-force password guessing against target accounts may cause account lockouts if a lockout threshold is set for those accounts and the guesses exceed that threshold.

•   Input fuzzing against web applications may cause performance issues for the application unless it is throttled not to exceed system capabilities during normal use.

•   Incident response resources may be consumed by investigation of a red team exercise in order to test response processes.

•   There may be loss of availability as a result of load testing or denial of service testing, if such testing is requested.

Target organizations should also specify whether some targets or data are more important than others as part of this impact analysis process. During testing, the tester will need to evaluate the impact of findings as they are discovered in order to determine whether escalation is required. Typically, testers will have a scale that defines the terms of impact, and this is discussed with the target audience prior to testing as part of reporting and scoping. A mutually understood grasp of impact and a strong communication escalation path help ensure timely response in the event of impact.

Findings must be addressed through a process called remediation. Target organizations have timelines associated with remediation activity. Organizations may choose to address the highest impact findings first, requiring escalation immediately upon identification rather than waiting for final reporting results.

Disclaimers

To protect the parties involved in the engagement, disclaimers may be added to the test plan, contracts, and reporting to clarify certain aspects of testing. A testing organization cannot claim a tester has identified all weaknesses that could ever exist in a target environment. Scope, testing, and time limitations, as well as differences in access, may prevent discovery of additional weaknesses. Environments may change over time, with new systems being added and configurations being changed, and each of these might introduce new vulnerabilities that can be exploited. Testing organizations will often include disclaimers to prevent penetration testing results from being interpreted as having any kind of warranty or guarantee for security.

Technical Constraints

Impacts to testing that are caused by technical limitations should be documented in the test plan and discussed in the report. Technical limitations are imposed by tools, budget concerns, and considerations of impact. Examples of technical limitations are

•   Certain types of attacks cannot be run in a third-party hosted environment, so the attacks used for testing are limited for all or part of the target environment.

•   Critical systems are known to be fragile and will crash if any unexpected traffic is sent to their open ports, so they are not allowed to be tested.

•   No tools are available to test denial of service attacks, and the budget for testing is not enough to cover acquisition of appropriate tooling for testing.

•   All testing must occur onsite, as systems are not accessible from outside and do not have access to external networks via any protocol.

Support Resources

Support resources are information that aids penetration testers in test planning and execution. Details about applications, systems, networks, facilities, staff, or operations may shape the test plan by identifying potential vectors for attack, required tooling for exploitation, and even aid in target selection. In some cases, these will be provided by the target organization. In other cases, such as black box testing, these will be discovered during the process of testing. You’ll find a few examples of support resources in Table 1.1-1. (Black box testing is discussed in Objective 1.3.)

TABLE 1.1-1   Example Support Resources

A web service is a web-accessible interface to a system. For another system to interact with that service, the service has expected inputs and expected outputs. Think of this as an API: a set of rules that determines what goes in, how, and what is expected to come out so that a client can interact with an application.

Since web applications are all programmed in different languages, it makes sense to use a common language that all of the application languages can read to process this input and output. One example is called XML (eXtensible Markup Language). This provides a uniform language to describe information from one web service to a client. If a web service does not provide XML data, it can be converted to XML for use. SOAP is a messaging specification for web services that defines how these web services can exchange that XML-structured information. A WSDL document describes what a client needs to know to interact with the web service. That document contains information like the protocol and port on which the service runs, what types of data are expected in a request, and what kinds of data the service can provide. So, when a client reads a WSDL, it can construct a request in XML, formulate it using a SOAP request, and use that to communicate with a web service.

Since WSDL can specify a port and protocol, it can be used outside of web application contexts. It has been expanded to broader capabilities. While it is still considered to be a lightweight implementation, it can be used in mail protocols, for example, too. WADL is specifically designed for web applications. It references web URIs, not ports and protocols. So, WSDL can do all that WADL can do, but WADL cannot do all WSDL can do. Some applications will use a WADL instead of a WSDL.

Web applications that are designed with architectural styles like REST may not use SOAP or XML at all. A RESTful API can be designed to exchange data universally using different data formats and protocols. Swagger allows developers to build documentation about how to access their APIs automatically from their code. Frequently, APIs have different versions, and Swagger will explain the differences in version automatically. Since it is much easier to use than WSDL or WADL, Swagger is a common format that is often considered to be the REST equivalent of a WSDL. WADL can also be used to describe a RESTful API; however, it is much harder to use and less flexible during ongoing development than Swagger.

Developers use software development kits (SDKs) to create applications. The SDK defines what functions are available for a language to interact with the underlying platform. The functions to interact with a Windows operating system versus a Linux operating system may differ, for example. Knowledge about the SDK will not only tell the tester what language is used to create the application but also what language functions and features are likely in use. This is great contextual information to search for vulnerabilities in applications.

REVIEW

Objective 1.1: Explain the importance of planning for an engagement   Penetration testers must gather information about their target in order to make an appropriate test plan. Keeping communications confidential and knowing when to communicate, what to communicate, and with whom to communicate are critical to the success of the test. Likewise, understanding budgetary impacts, technical limitations on testing, and working with stakeholders to plan for contingencies are all part of the penetration testing process. Following the rules of engagement, providing the right disclaimers, and escalating issues to the right parties protect the tester during the test.

1.1 QUESTIONS

1.   Who are the stakeholders for a pentesting engagement?

A.   The pentester, the project manager, and the CIO

B.   All of the security department

C.   Those who are affected by the testing

D.   The CEO, the CIO, the pentester, and an accounting staff member

2.   What does SOAP stand for?

A.   Service-Oriented Architecture Protocol

B.   Simple Object Access Protocol

C.   Serialized Object Authentication Protocol

D.   LSimple Object Architecture Protocol

3.   What is the difference between a WSDL, WADL, and Swagger file?

A.   WADL is for applications, WSDL is for services or applications, and Swagger can generate documentation from code.

B.   WSDL is for applications, Swagger is for documenting Java, and WADL is for services.

C.   WADL is for applications, Swagger is for documenting Java, and WSDL is for services.

D.   WSDL is for documenting Java, Swagger is for API documentation, and WADL is for applications.

4.   How does budget affect planning for an engagement?

A.   Budget is what a penetration test costs, and it determines what a penetration tester gets paid.

B.   Budget determines the scope, targets, and compliance requirements.

C.   Budget determines what tools a pentester can use and when the test takes place.

D.   Budget determines the tools and scope and influences how the test is conducted.

5.   What is an unknown requirement?

A.   Part of a threat model that addresses undiscovered contingencies.

B.   Documented limitations for the penetration tester.

C.   All of the things a penetration tester needs in order to test.

D.   Needs that were not predicted during planning.

6.   What is a technical constraint?

A.   Limitations on testing that are caused by technology.

B.   Scoping limitations on tested resources.

C.   Gaps in functionality of penetration testing tools.

D.   Limitations placed on technology to avoid scope creep.

7.   What is impact analysis?

A.   The results of gravity applied to an appliance.

B.   An exploration of a target organization’s risk acceptance processes.

C.   An examination of potential effects of testing on a target.

D.   Calculation of the time systems are down subsequent to testing.

8.   Name two kinds of disclaimers that may go along with a penetration test.

A.   Pentest disclaimer and scope disclaimer.

B.   Point-in-time disclaimer and completeness disclaimer.

C.   Impact disclaimer and liability disclaimer.

D.   Master services disclaimer and statement of work disclaimer.

9.   What is a communication escalation path, and why is it important?

A.   A chain of command that penetration testers use to communicate with authorized personnel. It introduces shared decision making and reduces liability to the tester.

B.   A list of people to call if others are not available. It ensures a fast response when a penetration test causes an incident.

C.   A procedural documentation of how communication occurs during testing. It avoids customer disputes by defining regular project status updates.

D.   A dispute resolution document that explains how disputes about testing are to be resolved through the chain of command. It makes sure penetration testers understand the testing scope.

10.   What does ROE stand for, and what is it?

A.   Rights, opportunities, and execution. A document that outlines penetration testing methodologies, expectations, and objectives.

B.   Remediation objectives for engagement. A document that outlines penetration testing objectives and explains remediation timelines.

C.   Rules of engagement. A document that explains how testing can be conducted, including rights and limitations for the tester.

D.   Rights, objectives, and engagement. A document that binds the target organization and the testing organization contractually to the work.

1.1 ANSWERS

1.   C   Stakeholders are those individuals who need to be involved in planning or communication because they are directly affected by the testing.

2.   B   Simple Object Access Protocol. See the section “Support Resources” for more details.

3.   A   WADL files are ideal for web applications, but not services, as they don’t specify ports or protocols. WSDL files may be for services or applications. Swagger files automatically generate API documentation from code and are commonly considered to be WSDL for RESTful APIs. See the section “Support Resources” for more details.

4.   D   A budget may introduce technical constraints and affect the target scope or type of testing. The “Budget” section discusses this in more detail.

5.   D   An unknown requirement is a requirement that was not known during initial planning based on the information available.

6.   A   Technical constraints are impacts to testing that are caused by technical limitations. Technical limitations are imposed by tools, budget concerns, and considerations of impact. See the section “Technical Constraints” for detailed examples.

7.   C   Impact analysis is a determination of the potential effects of testing on a target. See the section “Impact Analysis and Remediation Timelines” for details.

8.   B   Point-in-time disclaimer and completeness disclaimer. See the “Disclaimers” section for additional information.

9.   A   A communication escalation path is a chain of command that penetration testers use to communicate with authorized personnel. This protects the penetration tester against outcomes of these decisions, as they are not made solely by the penetration tester.

10.   C   ROE stands for rules of engagement, and it establishes how the testing can be conducted. This term is defined in the “Rules of Engagement” section.

Objective 1.2   Explain key legal concepts

A key separator between a penetration test and a jail sentence is the legal documentation for the test. Contracts, consent forms, and environmental considerations all need to be signed and documented as part of the penetration testing process. In this section, we’ll discuss the key legal concepts behind a penetration test.

Contracts

Contracts establish the mutual terms of interaction between the parties involved in the engagement. These documents define the rights and limitations for the client and the testing organization and should contain information that protects both parties. The documentation needed for a penetration test may take different formats, depending on whether the penetration tester is internal or external to the target organization. Consultancies, for example, must define terms of payment and confidentiality that internal penetration testers do not need, due to the nature of their employment for the organization being tested. Table 1.2-1 outlines key contract documents.

TABLE 1.2-1   Contracts

Cross-References

Environmental Differences

Penetration testers need to adhere not only to contract terms from the SOW, NDA, and MSA but also need to ensure that penetration testing activities conform to the laws, rules, and regulations of the geographical region and government, as well as the corporation being tested. A few examples of environmental restrictions are identified in Table 1.2-2.

TABLE 1.2-2   Environmental Restrictions

Written Authorization

One of the most important protections for a penetration tester—and a requirement for all penetration testing—is documented approval for testing. This should always be in writing, and it should be signed by the correct authority to grant approval. These documents should

•   Identify the appropriate authority to provide approval

•   Declare that the signature authority for the document has the authority to sign for the organization being tested

•   Define who is authorized to test

•   Declare what is being authorized

•   Describe the time frame for which authorization is being provided

In the case where third parties are involved—for example, cloud or hosting providers—additional authorization may need to be obtained from the third parties.

REVIEW

Objective 1.2: Explain key legal concepts   Appropriate documentation, such as an MSA, SOW, and NDA, may cover the initial agreement for an engagement, but further documentation is often needed to protect penetration testers from legal repercussions. It is also necessary to document approval from an authorized organizational representative and ensure the approval applies to the correct scope of work to cover the tester. Even when that is done, a tester needs to follow the rules, regulations, and laws according to local and national governments, corporate policies, and third parties.

1.2 QUESTIONS

1.   What are an SOW, NDA, and MSA?

A.   Statement of work, nondisclosure agreement, and material services authorization.

B.   Documents for penetration test reporting.

C.   Statute of work, nondisclosure agreement, and material services authorization.

D.   Statement of work, nondisclosure agreement, and master services agreement.

2.   How is an MSA different from an SOW?

A.   An MSA documents how two organizations should interact. An SOW describes the mutual expectations between two organizations for a specific engagement.

B.   An MSA describes the services and tests that will be conducted, how they will be conducted, and the goals. An SOW defines the laws, regulations, and policies that serve as guidelines for testing.

C.   An MSA is an accounting measure to track the balance sheet for a penetration test, while the SOW covers details of material acquisitions.

D.   An MSA documents how the SOW should be used to create the deliverable, including report templates, tools lists, and checklists. The SOW outlines the timeline for delivery.

3.   What are three other sources for laws, rules, or regulations that a penetration tester must follow outside of the contracts that govern an engagement?

A.   Countries, states, and cities.

B.   Laws and regulations, export restrictions, and policies.

C.   Laws and regulations, the stakeholders, and the penetration testing organization.

D.   Legislation, cities, and countries.

4.   What key information should an authorization agreement contain?

A.   Who is authorizing things, what is being authorized, and why it’s authorized.

B.   An authorized approver, an authorized tester, an authorized document, and an authorization process.

C.   A statement of authorization for the approver, what is being authorized, who is being authorized, and how long it’s authorized.

D.   A date, a signature, and a legal redline.

1.2 ANSWERS

1.   D   Statement of work, nondisclosure agreement, master services agreement. These documents and their respective purposes are discussed in the “Contracts” section.

2.   A   An MSA governs the overall relationship, including payment terms, indemnity, and dispute resolution. An SOW is specific to the engagement and includes milestones, payment schedule, costs, and details related to the scope of a particular project.

3.   B   Local and government laws and regulations, corporate policies, and export restrictions. Limitations are discussed in the “Environmental Differences” section.

4.   C   A declaration that the person signing the document has authority to grant approval on behalf of the organization, what is being tested, who is testing, what testing is being approved, and the length of time the approval is granted. The “Written Authorization” section addresses this.

Objective 1.3   Explain the importance of scoping an engagement properly

Target organizations have a budget that must be met for testing. Testing organizations need to balance available resources with their ability to produce deliverables that meet the target organization’s needs. Scoping is the process of defining the engagement goals, deliverables, tasks, resources, costs, and deadlines in order to meet the budget and testing capabilities. Some considerations for scoping are

•   Goals of testing, to determine testing type

•   Any special considerations that affect how the test is run

•   Targets for testing

•   Technical considerations for testing

•   Testing strategy

•   Tolerance to impact and risk acceptance by the target organization

•   Scheduling for testing

•   Threat actor modeling

Much of this information is collected during planning, as referenced in Objective 1.1. The scoping process formalizes these details for the creation of the SOW.

Types of Penetration Testing

Understanding why an organization desires to have a penetration test will enable a tester to determine the appropriate type of assessment. If the primary objective of an organization is to obtain compliance, then the test must follow the rules defined by the regulation or the laws with which the organization intends to comply. If an organization wishes to test its incident response staff, engaging a red team may be most appropriate. Some organizations do not need a penetration test at all, but may find a vulnerability assessment more appropriate. Gathering this information up-front helps the tester make certain to deliver the right test for the organization. In Table 1.3-1, you will find a summary of testing types.

TABLE 1.3-1   Testing Types

Goals-Based/Objectives-Based Penetration Testing

An organization may establish a goal-oriented penetration test by identifying data or assets that they consider most sensitive. As an example, a company that invents things may consider the intellectual property behind its inventions to be critical to the success of the business. If someone were to break into the company and steal that information, the company may lose business (or go out of business) because they are no longer able to capitalize on the proprietary nature of the invention. In this case, a penetration tester may construct a test to prove whether it is possible to access that information and get it out of the company’s environment.

Compliance-Based Penetration Testing

Certain laws, rules, and regulations define requirements that some kinds of penetration testing are designed to explore. Compliance requirements might dictate the types of data that must be protected, how it may be handled by a tester, what represents a finding when data is exposed, and even where testing must occur logically within the target environment. All of these must be considered during test planning and execution if compliance testing is the goal. (Compliance-based penetration testing is discussed in more detail in Objective 1.4.)

Red Team Testing

Unlike other types of penetration testing, the goal of red team testing often includes stealth. The goal of standard penetration testing is not to avoid detection, but to evaluate impact if attacks are successful. These tests are often used to evaluate detection capabilities and response efficacy of organizations. Goals that drive red team engagements may include things like specific dwell time (the amount of time the tester maintains access within the target environment without being ejected by defense) or even goal-oriented considerations like the compromise of a specific system, account, application, or process within the target environment while remaining undetected.

Special Scoping Considerations

Organizations that need penetration testing as part of merger activities and evaluations of supply chains may have special testing considerations. Premerger testing is due diligence activity designed to highlight potential impacts that may be realized during the merger or after the merger is complete. Test results should identify countermeasures to prevent breach. Supply chain testing examines data via the business processes through which it flows. This may involve testing business partners, service providers, and other vendors who participate in the business process to be tested. As such, stakeholder identification and approval may require special considerations.

Target Selection

Targets should be defined as part of the penetration test scope. This ensures that the time and resources allocated for testing are appropriate, as well as providing a scope for testing authorization. Targets should be selected according to the target organization’s goals for testing, and the tester should identify any special considerations for testing those targets as part of the scoping process during target identification.

Targets

The testing vector will be determined by the organization’s goals. A test from outside of the organization’s environment may emulate an Internet-based attacker, for example. A test from inside the organization’s environment may explore the impact of a workstation that has been compromised by malware or is being used by a malicious insider, for example. Organizations may desire only to test the security of web applications, or of networks, or only the security of their physical controls. All of these will be considered when selecting targets.

Onsite and offsite targets are important considerations during physical testing. Planning a physical test to get hands-on access to an asset, for example, requires knowledge of where an asset physically resides. An onsite asset resides in the same building or physical location as where the test will occur. An offsite asset resides in some other location.

Internal targets are accessible from inside the organization, but not from outside of it. These targets are often tested as an evaluation of impact from insider threats. However, these are not always onsite assets. Assets that are hosted in a remote data center may still be internally accessible. Internal targets often have access to organizationally sensitive data, processes, or other systems. Access to internal targets typically implies that an attacker has bypassed perimeter security controls, such as firewalls or locked doors.

External targets are Internet facing, or publicly facing. These can be reached from outside the organization’s network or facilities. These are not always offsite assets. Assets that are hosted in the same building as internal staff may be externally accessible.

First-party and third-party hosted targets may have different requirements for testing authorization. A first-party hosted target is run by the organization that uses it. A third-party hosted target may be run by a different organization than the organization that uses the target’s services. For third-party hosted targets, targets may require third-party testing authorization. Organizations may prefer third-party solutions when it is perceived that an external organization can better focus on securing and maintaining the target.

Physical targets are devices or assets that can be touched. This does not describe data or records, but computers, laptops, USB drives, and door locks. These are common targets of theft attacks (for example, shoplifting), injection of malware via direct access to the system, and introduction of attack devices. With direct access to a physical target that is internal, a tester may be able to get access to other internal targets.

Users have been historically considered the weakest link in the security chain. As technical controls have evolved and become more difficult to bypass, attackers have learned to target users with social engineering techniques in order to exploit access to internal targets that users already possess.

Cross-References

SSIDs are the names of wireless local area networks (WLANs). Wireless networks may be accessed without a physical connection, and may often be accessed nearby and from outside the physical controls of an organization. If an organization broadcasts SSIDs, they may be more vulnerable to wireless attacks.

Cross-References

Applications often contain sensitive information or are connected to other systems that contain sensitive information. One example is credit card data. Websites may be designed to sell goods and process a credit card via a web application. That credit card information must pass through the application or may even be in a database that the application talks to. External applications may be particularly attractive targets for attackers due to the information and access they have.

Testing Considerations

Once target selection is complete, the testing plan needs to consider the role of organizational controls during testing. Knowing about the security controls in the environment helps the tester and the target organization realize the most gains, given the time allocated for testing. Organizational policies and technical controls are both security controls.

Some examples of organizational policies may include things like password length and complexity requirements, rules about account reuse, or rules governing what assets are allowed to be deployed within the environment. Testers must consider the impact of these rules on testing, as testing must follow all organizational policies. Additionally, assets that do not adhere to policies may introduce exploitable vulnerabilities whose impact should be explored.

Technical controls are also important for test scoping. Testers may use knowledge about technical controls to plan what kinds of attacks are done during testing, determine the deployment location for testing, and define details about the attack vector. To use the time most effectively, organizations may enable a tester to bypass certain technical controls during testing. This focuses the test on exploring the impact of an attack on the protected targets rather than exploring the possibility of whether the control can be bypassed. Three examples of where a penetration test may need to bypass a control are IPS, WAF, and NAC.

IPSs and WAFs examine transactions over the network, detect patterns of attack, and then block them. To prevent this from stopping a penetration tester whose goal is not to be stealthy, the target organization may choose to whitelist the penetration testing platform so that it is not analyzed by or blocked by these controls.

Another example of controls that may feature prominently during a scoping discussion is network access control (NAC). NAC checks the security posture of a device when it connects to a network and either enables it to connect or prevents it from connecting to the network. In the case where only approved devices are whitelisted, the target organization and the tester must decide whether time should be dedicated to bypassing the NAC control during testing or if an exception should be placed to allow the penetration testing device onto the network so that the time can be spent evaluating other things.

The process of temporarily or permanently allowing the bypass of a security control creates a security exception. A security exception allows fragile systems that cannot be patched due to stability issues to remain unpatched even though a policy requires updating, for example. Security exceptions are often managed through a risk management process that requires security review and approval within the organization. Often, this review process requires exceptions to implement additional controls to mitigate the risks posed. An example might be limiting the exception to a very narrow range of systems (one system excepted) or to limit the exception to a narrow time frame. In the cases where a system has a security exception, it may have known vulnerabilities. However, exploiting those vulnerabilities has no value during a penetration test. Knowing what security exceptions exist and whether those fall within the scope of testing should help avoid unnecessary disruption and unnecessary work. (Read more in the “Risk Acceptance” and “Tolerance to Impact” sections of this objective.)

Strategy

To get an “attacker’s point of view,” many organizations seeking penetration testing will initially opt for black box security testing. By providing the tester with no information at all, an organization may gain some insight into how attackers find what is interesting to attack. However, since this approach requires significant interaction with the owner—a tester cannot test something unless it is approved to be in scope, so that must be verified along the way—some organizations opt for a gray box security testing approach instead. During this type of test, some information may be disclosed to the tester, such as network ranges and specific system details—just enough to get the test going without needing verification at each step.

Both of these approaches are good for an initial idea about what an attacker might see. However, the reality is that attackers are not as limited by time as penetration testers. What a penetration test may find during a limited period of time is not as broad as what an attacker might identify with months of time to perform a similar examination. Therefore, it is common for organizations to follow black box testing with gray box or even white box security testing, where more complete knowledge of the target (and access to the target) enables the penetration tester to more thoroughly explore the target during a shorter time frame. These strategies are summarized in Table 1.3-2.

TABLE 1.3-2   Testing Strategies

Risk Acceptance

In the event that a risk is identified, the organization must decide how to deal with the risk. An organization may accept the risk, avoid the risk, transfer the risk, or mitigate it. In the case a risk is accepted, it has been determined that nothing further needs to be done about the risk. Avoiding the risk involves eliminating the risk, for example, by fixing the vulnerability entirely or removing the asset that introduced the risk. Risk transference is the process of getting someone else to take responsibility for the risk, such as using insurance policies or outsourcing. Mitigating a risk is reducing the likelihood of impact, or reducing the impact itself until it is tolerable.

Tolerance to Impact

Penetration testing can cause impact. Whether this is an impact to performance, the unintentional disruption of a service, or even the triggering of alarms on monitoring systems, the impact of penetration testing needs to be discussed as part of scoping. The target organization will want to discuss what they are willing to tolerate so that the appropriate communication, timing, and testing techniques are used. This will involve documenting what testing is in scope and what testing is out of scope as part of the scoping process.

Scheduling

Depending on the target organization’s tolerance to impact, certain types of testing may need to be conducted either during off-hours or during times when staff will be known to be available to respond. Anything that is likely to cause a denial of service, for example, may need to be scheduled outside of business hours to lessen the impact.

Scope Creep

Once the scope is established and the SOW is signed, the test is bound by contractual agreement. Failure to follow this agreement could waive protections that the SOW grants to the tester. Any changes to what is tested or to the testing being done would take away from the already planned activities. Deviations from the SOW can cause disputes based on differences between what is done versus what was agreed upon in a signed agreement. It may cause the tester or target organization to incur additional unplanned costs. It could affect the quality of the test if the amount of work is increased without increasing the time or resources allocated to perform the work.

When these are requested by the target organization, this is called scope creep. Up-front planning is vital for avoiding scope creep. If additional testing is requested or the scope is significantly changed, the SOW may need to be addended with additional language and approvals.

Threat Actors

Threat actors are those who are willing to attack an organization with the intent to do harm. Capabilities and intents vary. Threat intelligence organizations have different frameworks to describe these attributes. For penetration testing, it may be necessary to explore the impact of a particular threat actor within an environment. This may influence the attack vector (internal or external), techniques for achieving access, tooling used, or goals of testing. Each actor is different, but Table 1.3-3 gives examples of capabilities and intents in broad strokes for some identified adversary types.

TABLE 1.3-3   Threat Actors, Capabilities, and Intent

Some organizations break threat actors into tiers based on their capability and ability to cause impact. Generally, higher-tier actors use a blend of attacks to obtain access, have greater expertise to execute attacks, are capable of bypassing more advanced defenses, and have significant financial or organizational backing. Lower-tier actors tend to use whatever tools are available, focus on opportunistic attacks, have the least experience, are often thwarted by automatic security controls, and have minimal financial or organizational backing.

APTs are often highly skilled with significant financial and technological resources that can be used to stage complex, low-visibility attacks. Frequently backed by political or criminal organizations, they may seek to conduct espionage or financial heists with the intent of obtaining financial gain, political or economic advantage, or to influence current events.

Script kiddies are unskilled attackers who execute opportunistic automated attacks using off-the-shelf technology, often without understanding their use. These often exist in higher numbers than other tiers of attacker and tend to be responsible for denial of service, web defacement, or other forms of Internet vandalism. Script kiddies often work alone or in small disorganized groups.

Hacktivists are often politically motivated, intermediately skilled individuals whose disaffection with a company, social element, or entity causes them to hack a target with the goal of embarrassing or disrupting it. These individuals tend to operate alone or in small semi-organized groups.

Insider threat is a term used to describe individuals with existing access or knowledge from inside a target organization. These may be disgruntled staff who wish to get even by causing harm to their organization, victims of blackmail or other forms of extortion, or even innocent bystanders who fall for social engineering schemes of external attackers.

Threat Models

Threat modeling is a structured process that allows organizations to quantify and enumerate risks by identifying attack vectors, attack methods, and tools that might be used against a target environment. The focus here is on what actions could happen to which assets. There are several frameworks and methodologies to do this, and additional resources are cited here if you would like to read more about some of the tooling used in threat modeling. But the key point of threat modeling is to define the appropriate scope for testing by understanding what assets are most important and which actions are most likely to achieve the goals of the test.

REVIEW

Objective 1.3: Explain the importance of scoping an engagement properly   Scoping an engagement properly is vital to the success of an engagement. Appropriate scoping helps prevent disputes by avoiding scope creep and handling it appropriately when it arises. Not only does this help ensure that testing activities remain within budget, but it helps minimize impact and align testing goals with the target organization’s goals. Scoping involves the selection of targets, the appropriate testing type, and testing strategies based on the goals of the organization. It involves scheduling testing appropriate to the target organization’s tolerance for impact, understanding threat actors and organizational threat models, and accounting for any special considerations that affect the scope.

1.3 QUESTIONS

1.   An APT would typically be considered to be what threat actor tier?

A.   High tier.

B.   Low tier.

C.   Mid-tier.

D.   No tier.

2.   Focusing on what actions may happen to what assets is part of what process?

A.   Penetration test planning.

B.   Threat modeling.

C.   Scoping.

D.   Target selection.

3.   Name three types of penetration testing.

A.   Vulnerability assessment, validated penetration testing, full-scope testing.

B.   Goals-based or objectives-based penetration testing, red teaming, and compliance-based penetration testing.

C.   Black box, gray box, and white box penetration testing.

D.   Compliance-oriented penetration testing, goal-teaming or objective-teaming penetration testing, and red-blue penetration testing.

4.   What is the difference between black box penetration testing and gray box penetration testing?

A.   Black box testing is only done by criminals, but gray box testing has approval.

B.   Black box testing does not define the scope or testing method in advance of the test. Gray box testing allows hands-on access to all assets.

C.   Black box testing gives no information to the tester. Gray box testing gives some (but not all) information to the tester.

D.   Black box testing requires a full red team, but gray box testing can be done by a single penetration tester with an auditor.

5.   Why are applications targets for testing?

A.   They tend to contain or have access to lucrative data.

B.   They’re always exposed on the Internet, so they’re easy to access from any vector.

C.   They’re easier to attack than anything else.

D.   OWASP requires organizations with applications to have penetration tests.

6.   What kind of attack is most often used against users?

A.   APT.

B.   Physical attacks.

C.   Lateral movement.

D.   Social engineering.

7.   Name three controls discussed in this text where whitelisting may be relevant during a penetration test.

A.   Gray box testing, black box testing, and white box testing.

B.   Policies, regulations, and export restrictions.

C.   NAC, IPS, and WAF.

D.   Firewalls, IP ranges, and password vaults.

8.   When would testing that may cause a denial of service condition typically be scheduled?

A.   At lunch.

B.   On the weekend.

C.   Outside of business hours.

D.   When all hands can be on deck.

9.   What threat actor type is most known for only using tools that other people have made?

A.   Script kiddies.

B.   APTs.

C.   Hacktivists.

D.   Social engineers.

10.   Name four ways risk may be handled.

A.   Risk jumping, risk passing, risk deference, and risk aversion.

B.   Risk avoidance, risk mitigation, risk transference, and risk acceptance.

C.   Risk tolerance, risk calculation, risk adjustment, and risk management.

D.   Risk workgroups, risk juggling, risk actuaries, and risk journalism.

1.3 ANSWERS

1.   A   APTs are typically actors of the higher tiers, due to their advanced capabilities, resources, and custom attacks.

2.   B   Threat modeling is a structured process that allows organizations to quantify and enumerate risks by identifying attack vectors, attack methods, and tools that may be used against a target environment. The focus here is on what actions could happen to which assets. Threat modeling is discussed in detail in the “Threat Actors” section.

3.   B   Goals-based or objectives-based penetration testing, red teaming, and compliance-based penetration testing. These are discussed in more detail in the “Types of Penetration Testing” section. Black box, gray box, and white box testing are testing strategies.

4.   C   Black box testing involves no information about the targets of the test, whereas gray box testing involves some (but not full) information about the targets of testing. Testing strategies are discussed in the “Strategy” section.

5.   A   Attackers may target applications because of the value of the information they contain or have access to. Therefore, choosing applications as a testing target to evaluate their security may be important to an organization that values the information the applications contain.

6.   D   Social engineering is the process of getting a person to do something that he or she would not normally do or want to do, with the interest of gaining additional access or accomplishing other attack goals. The “Target Selection” section discusses users as a target.

7.   C   NAC, IPS, and WAF. Each of these controls is designed to block or prevent attack activity, often as a first line of defense. Organizations may desire (or be required to have) testing of other security layers.

8.   C   If such attacks are in scope, a target organization may ask these to be scheduled outside of business hours.

9.   A   Script kiddies reside in the lowest adversary tier due to their average skill level. They are known for relying exclusively on tooling that others have made, often without understanding the tool.

10.   B   Risk avoidance, risk mitigation, risk transference, and risk acceptance. These terms are discussed in the “Risk Acceptance” section.

Objective 1.4   Explain the key aspects of compliance-based assessments

Compliance-based assessments may have special requirements, limitations, or caveats over other kinds of penetration testing. To ensure that you consider these during penetration test planning and execution, the following sections will discuss fundamental differences between compliance-based assessment and other kinds of penetration testing. This includes discussion about the focus of compliance-based assessments, compliance frameworks, and regulations.

Compliance-Based Assessments, Limitations, and Caveats

Unlike other kinds of penetration testing where the testing strategy and methodology may be solely determined by discussions between the tester and the stakeholders, compliance-based assessments have regulated requirements. Regulations and compliance frameworks may

•   Dictate rules to complete the assessment (what should be tested and how)

•   Require evaluation of the password policies (secure key and password handling and transmission)

•   Define testing methods to evaluate data isolation and data handling

•   Place limitations on network or storage access

Therefore, the questions that need to be asked during information gathering and scoping may be significantly different than with other types of testing.

Rules to Complete Assessment

The objectives for a penetration test will be defined by the regulation or compliance framework. The regulation may also place stipulations on how the test can be conducted. There are several compliance frameworks, for example, CoBIT, CISQ, FedRAMP, ISO, and NIST. Compliance frameworks may help guide scoping and test planning, with checklists for security configurations that should be validated with penetration testing. Each uses different terminology and evaluates security according to different criteria. So, if a target organization uses a specific framework, the report delivery may need to consider the language and guidelines of a particular format.

Password Policies and Key Management

Many regulations focus on testing password policies, including secure key storage and management, password storage and management, as well as password length and complexity. Organizations with this requirement may need tests that search for passwords or keys that are insecurely stored or transmitted, or the use of insecure encryption protocols.

Data Isolation

PCI-DSS, for example, defines a card processing environment as a subset of the target organization’s environment and places explicit requirements on where a penetration testing platform must be deployed in relation to that environment for testing. Making sure that data cannot be exfiltrated or transmitted across boundaries or that it can’t be intercepted in transit may be required as part of the test.

Limitations

The regulation or framework may stipulate data handling restrictions. This includes how the tester sanitizes a testing appliance after data is accessed, how accessed data is used, and even what data the tester is allowed to access. As an example, regulations may require a tester to use methods to demonstrate that it is possible to access a database containing cardholder data or other data that is legally protected without actually displaying the records. Regulations may also require reports to include no unredacted protected data and that penetration testers not store evidence that includes data after the testing period has ended.

The network access scope may also be limited by regulation. What must be tested—and what can be tested—is often defined by regulations. Going against this may invalidate the test as a mechanism for asserting compliance for required penetration tests.

Clearly Defined Objectives Based on Regulations

There are numerous regulated requirements and governing standards that may require compliance, such as PCI-DSS, SOX, HIPAA, ISO, and DISA-STIGs. Some of these are international, and others are specific to the United States. Depending on the region and industry in which an organization operates, a penetration test may be required to assess compliance with any one or more of these standards—or others, such as GDPR (the EU General Data Protection Regulation). Each regulation specifies different requirements for tester qualifications, assessment goals, and data handling that will define the objectives of the test.

REVIEW

Objective 1.4: Explain the key aspects of compliance-based assessments   Compliance-based penetration tests require different information from the target organization and a specific understanding of the guiding regulation or framework. The framework or regulation will determine how the test should be conducted, as well as establishing the specific goals of testing. The goals of testing will often focus on data isolation and secure data transmission, as well as secure password and key management, and will stipulate limitations about storage and network access that will often be distinct from the requirements of other testing types. The approach to information gathering, scoping, and planning for compliance-based penetration tests is therefore significantly differentiated from that of other testing types.

1.4 QUESTIONS

1.   What does PCI-DSS stand for?

A.   Personal Computing Initiative – Defense Strategy Service.

B.   Payment Card Industry – Data Security Standard.

C.   Protected Confidential Information – Data Security Standard.

D.   Penetration Certification Industry – Detainment Security Solutions.

2.   What may make a compliance-based penetration test different from a standard goals-based penetration test?

A.   Regulation defines testing objectives and may have additional limitations on data storage and network access, as well as password/key storage and data management.

B.   Compliance-based tests require a checklist, and goals-based penetration tests are purely improvisational.

C.   Compliance-based penetration tests are purchased by the government, not the organization that holds the data, so special third-party approvals are required that goals-based tests don’t need.

D.   Compliance-based penetration tests require the penetration tester to be compliant with all laws and regulations, whereas goal-oriented tests only require that the tester follow the SOW.

3.   What does a compliance framework do?

A.   Compliance frameworks ensure compliance.

B.   Compliance frameworks provide penetration testing checklists, including tools that must be used and the syntax of tests that must be run.

C.   Compliance frameworks provide legal language that must be used in the SOW or MSA, and are tools used exclusively by the legal team.

D.   Compliance frameworks may provide goals for testing, guidance for testing methods, and language to express results.

1.4 ANSWERS

1.   B   Payment Card Industry – Data Security Standard. This is a compliance standard referenced in the “Compliance-Based Assessments, Limitations, and Caveats” section.

2.   A   Regulations, standards, and policies may impose limitations on how testing can be conducted (including data and network access limitations) and require specific testing objectives (such as focusing on secure storage and management for keys and passwords, and data isolation). The “Compliance-Based Assessment, Limitations, and Caveats” section discusses each of these concepts in further detail.

3.   D   Much like regulations, compliance frameworks may define goals for testing and influence testing methods, as well as describe language to be used to express results. This is discussed in the “Compliance-Based Assessment, Limitations, and Caveats” section.