Why Do We Need to Do Penetration Testing?

So, why do we need penetration testing? Well, first of all, as someone who is responsible for securing and defending a network/system, you want to find any possible paths of compromise before the bad guys do. For years we have developed and implemented many different defensive techniques (for instance, antivirus, firewalls, intrusion prevention systems [IPSs], anti-malware). We have deployed defense-in-depth as a method to secure and defend our networks. But how do we know if those defenses really work and whether they are enough to keep out the bad guys? How valuable is the data that we are protecting, and are we protecting the right things? These are some of the questions that should be answered by a penetration test. If you build a fence around your yard with the intent of keeping your dog from getting out, maybe it only needs to be 4 feet tall. However, if your concern is not the dog getting out but an intruder getting in, then you need a different fence—one that would need to be much taller than 4 feet. Depending on what you are protecting, you might also want razor wire on the top of the fence to deter the bad guys even more. When it comes to information security, we need to do the same type of assessments on our networks and systems. We need to determine what it is we are protecting and whether our defenses can hold up to the threats that are imposed on them. This is where penetration testing comes in. Simply implementing a firewall, an IPS, anti-malware, a VPN, a web application firewall (WAF), and other modern security defenses isn’t enough. You also need to test their validity. And you need to do this on a regular basis. As you know, networks and systems change constantly. This means the attack surface can change as well, and when it does, you need to consider reevaluating the security posture by way of a penetration test.

Threat Actors

Before you can understand how an ethical hacker or penetration tester can mimic a threat actor (or malicious attacker), you need to understand the different types of threat actors. The following are the most common types of malicious attackers we see today:

• Organized crime: Several years ago, the cybercrime industry took over the number-one spot, previously held by the drug trade, for the most profitable illegal industry. As you can imagine, it has attracted a new type of cybercriminal. Just as it did back in the days of Prohibition, organized crime goes where the money is. Organized crime consists of very well-funded and motivated groups that will typically use any and all of the latest attack techniques. Whether that is ransomware or data theft, if it can be monetized, organized crime will used it.

• Hacktivists: This type of threat actor is not motivated by money. Hactivists are looking to make a point or to further their beliefs, using cybercrime as their method of attack. These types of attacks are often carried out by stealing sensitive data and then revealing it to the public for the purpose of embarrassing or financially affecting a target.

• State-sponsored attackers: Cyber war and cyber espionage are two terms that fit into this category. Many governments around the world today use cyber attacks to steal information from their opponents and cause disruption. Many believe that the next Pearl Harbor will occur in cyberspace. That’s one of the reasons the United States declared cyberspace to be one of the operational domains that U.S. forces would be trained to defend.

• Insider threats: An insider threat is a threat that comes from inside an organization. The motivations of these types of actors are normally different from those of many of the other common threat actors. Insider threats are often normal employees who are tricked into divulging sensitive information or mistakenly clicking on links that allow attackers to gain access to their computers. However, they could also be malicious insiders who are possibly motivated by revenge or money.

Why Do We Need to Follow a Methodology for Penetration Testing?

As just mentioned, scope creep is one reason for utilizing a specific methodology; however, there are many other reasons. For instance, when performing a penetration test for a customer, you must show that the methods you plan to use for testing are tried and true. By utilizing a known methodology, you are able to provide documentation of a specialized procedure that has been used by many people.

Environmental Considerations

There are, of course, a number of different types of penetration tests. Often they are combined in the overall scope of a penetration test; however, they can also be performed as individual tests as well.

The following is a list of some of the most common environmental considerations for the types of penetration tests today:

• Network infrastructure tests: Testing of the network infrastructure can mean a few things. For the purposes of this book, we say it is focused on evaluating the security posture of the actual network infrastructure and how it is able to help defend against attacks. This often includes the switches, routers, firewalls, and supporting resources, such as authentication, authorization, and accounting (AAA) servers and IPSs. A penetration test on wireless infrastructure may sometimes be included in the scope of a network infrastructure test. However, additional types of tests beyond a wired network assessment would be performed. For instance, a wireless security tester would attempt to break into a network via the wireless network either by bypassing security mechanisms or breaking the cryptographic methods used to secure the traffic. Testing the wireless infrastructure helps an organization to determine weaknesses in the wireless deployment as well as the exposure. It often includes a detailed heat map of the signal disbursement.

• Application-based tests: This type of pen testing focuses on testing for security weaknesses in enterprise applications. These weaknesses can include but are not limited to misconfigurations, input validation issues, injection issues, and logic flaws. Because a web application is typically built on a web server with a back-end database, the testing scope normally includes the database as well. However, it focuses on gaining access to that supporting database through the web application compromise. A great resource that we mention a number of times in this book is the Open Web Application Security Project (OWASP).

• Penetration testing in the cloud: Cloud service providers (CSPs) such as Azure, Amazon Web Services (AWS), and Google Cloud Platform (GCP) have no choice but to take their security and compliance responsibilities very seriously. For instance, Amazon created the Shared Responsibility Model to describe the AWS customers’ responsibilities and Amazon’s responsibilities in detail (see https://aws.amazon.com/compliance/shared-responsibility-model). The responsibility for cloud security depends on the type of cloud model (software as a service [SaaS], platform as a service [PaaS], or infrastructure as a service [IaaS]). For example, with IaaS, the customer (cloud consumer) is responsible for data, applications, runtime, middleware, virtual machines (VMs), containers, and operating systems in VMs. Regardless of the model used, cloud security is the responsibility of both the client and the cloud provider. These details need to be worked out before a cloud computing contract is signed. These contracts vary depending on the security requirements of the client. Considerations include disaster recovery, service-level agreements (SLAs), data integrity, and encryption. For example, is encryption provided end to end or just at the cloud provider? Also, who manages the encryption keys—the CSP or the client? Overall, you want to ensure that the CSP has the same layers of security (logical, physical, and administrative) in place that you would have for services you control. When performing penetration testing in the cloud, you must understand what you can do and what you cannot do. Most CSPs have detailed guidelines on how to perform security assessments and penetration testing in the cloud. Regardless, there are many potential threats when organizations move to a cloud model. For example, although your data is in the cloud, it must reside in a physical location somewhere. Your cloud provider should agree in writing to provide the level of security required for your customers. As an example, the following link includes the AWS Customer Support Policy for Penetration Testing: https://aws.amazon.com/security/penetration-testing.

When talking about penetration testing methods, you are likely to hear the terms unknown-environment (previously known as black-box), known-environment (previously known as white-box), and partially known environment (previously known as gray-box) testing. These terms are used to describe the perspective from which the testing is performed, as well as the amount of information that is provided to the tester:

• Unknown-environment test: In an unknown-environment penetration test, the tester is typically provided only a very limited amount of information. For instance, the tester may be provided only the domain names and IP addresses that are in scope for a particular target. The idea of this type of limitation is to have the tester start out with the perspective that an external attacker might have. Typically, an attacker would first determine a target and then begin to gather information about the target, using public information, and gain more and more information to use in attacks. The tester would not have prior knowledge of the target’s organization and infrastructure. Another aspect of unknown-environment testing is that sometimes the network support personnel of the target may not be given information about exactly when the test is taking place. This allows for a defense exercise to take place as well, and it eliminates the issue of a target preparing for the test and not giving a real-world view of how the security posture really looks.

• Known-environment test: In a known-environment penetration test, the tester starts out with a significant amount of information about the organization and its infrastructure. The tester would normally be provided things like network diagrams, IP addresses, configurations, and a set of user credentials. If the scope includes an application assessment, the tester might also be provided the source code of the target application. The idea of this type of test is to identify as many security holes as possible. In an unknown-environment test, the scope may be only to identify a path into the organization and stop there. With known-environment testing, the scope is typically much broader and includes internal network configuration auditing and scanning of desktop computers for defects. Time and money are typically deciding factors in the determination of which type of penetration test to complete. If a company has specific concerns about an application, a server, or a segment of the infrastructure, it can provide information about that specific target to decrease the scope and the amount of time spent on the test but still uncover the desired results. With the sophistication and capabilities of adversaries today, it is likely that most networks will be compromised at some point, and a white-box approach is not a bad option.

• Partially known environment test: A partially known environment penetration test is somewhat of a hybrid approach between unknown- and known-environment tests. With partially known environment testing, the testers may be provided credentials but not full documentation of the network infrastructure. This would allow the testers to still provide results of their testing from the perspective of an external attacker’s point of view. Considering the fact that most compromises start at the client and work their way throughout the network, a good approach would be a scope where the testers start on the inside of the network and have access to a client machine. Then they could pivot throughout the network to determine what the impact of a compromise would be.

Surveying Different Standards and Methodologies

There are a number of penetration testing methodologies that have been around for a while and continue to be updated as new threats emerge.

The following is a list of some of the most common penetration testing methodologies and other standards:

• MITRE ATT&CK: The MITRE ATT&CK framework (https://attack.mitre.org) is an amazing resource for learning about an adversary’s tactics, techniques, and procedures (TTPs). Both offensive security professionals (penetration testers, red teamers, bug hunters, and so on) and incident responders and threat hunting teams use the MITRE ATT&CK framework today. The MITRE ATT&CK framework is a collection of different matrices of tactics, techniques, and subtechniques. These matrices—including the Enterprise ATT&CK Matrix, Network, Cloud, ICS, and Mobile—list the tactics and techniques that adversaries use while preparing for an attack, including gathering of information (open-source intelligence [OSINT], technical and people weakness identification, and more) as well as different exploitation and post-exploitation techniques. You will learn more about MITRE ATT&CK throughout this book.

• OWASP Web Security Testing Guide (WSTG): The OWASP WSTG is a comprehensive guide focused on web application testing. It is a compilation of many years of work by OWASP members. OWASP WSTG covers the high-level phases of web application security testing and digs deeper into the testing methods used. For instance, it goes as far as providing attack vectors for testing cross-site scripting (XSS), XML external entity (XXE) attacks, cross-site request forgery (CSRF), and SQL injection attacks; as well as how to prevent and mitigate these attacks. You will learn more about these attacks in Chapter 6, “Exploiting Application-Based Vulnerabilities.” From a web application security testing perspective, OWASP WSTG is the most detailed and comprehensive guide available. You can find the OWASP WSTG and related project information at https://owasp.org/www-project-web-security-testing-guide/.

• The National Institute of Standards and Technology (NIST) Special Publication (SP) 800-115: NIST SP 800-115 is a document created by NIST (part of the U.S. Department of Commerce) for the purpose of providing organizations with guidelines on planning and conducting information security testing. It superseded the previous standard document, SP 800-42. SP 800-115 is considered an industry standard for penetration testing guidance and is called out in many other industry standards and documents. You can access NIST SP 800-115 at https://csrc.nist.gov/publications/detail/sp/800-115/final.

• Open Source Security Testing Methodology Manual (OSSTMM): The OSSTMM, developed by Pete Herzog, has been around a long time. Distributed by the Institute for Security and Open Methodologies (ISECOM), the OSSTMM is a document that lays out repeatable and consistent security testing. It is currently in version 3, and version 4 is in draft status. The OSSTMM has the following key sections:

  • Operational Security Metrics

  • Trust Analysis

  • Work Flow

  • Human Security Testing

  • Physical Security Testing

  • Wireless Security Testing

  • Telecommunications Security Testing

  • Data Networks Security Testing

  • Compliance Regulations

  • Reporting with the Security Test Audit Report (STAR)

The OSSTMM can be found at https://www.isecom.org.

• Penetration Testing Execution Standard (PTES): PTES provides information about types of attacks and methods, and it provides information on the latest tools available to accomplish the testing methods outlined. PTES involves seven distinct phases:

  • Pre-engagement interactions

  • Intelligence gathering

  • Threat modeling

  • Vulnerability analysis

  • Exploitation

  • Post-exploitation

  • Reporting

  For more information about PTES, see http://www.pentest-standard.org.

• Information Systems Security Assessment Framework (ISSAF): The ISSAF is another penetration testing methodology similar to the others on this list with some additional phases. ISSAF covers the following phases:

  • Information gathering

  • Network mapping

  • Vulnerability identification

  • Penetration

  • Gaining access and privilege escalation

  • Enumerating further

  • Compromising remote users/sites

  • Maintaining access

  • Covering the tracks

Requirements and Guidelines for Penetration Testing Labs

Now let’s dig a bit deeper into what a penetration testing lab environment might look like and some best practices for setting up such a lab. The following is a list of requirements for a typical penetration testing environment:

• Closed network: You need to ensure controlled access to and from the lab environment and restricted access to the Internet.

• Virtualized computing environment: This allows for easy deployment and recovery of devices being tested.

• Realistic environment: If you are staging a testing environment, it should match the real environment as closely as possible.

• Health monitoring: When something crashes, you need to be able to determine why it happened.

• Sufficient hardware resources: You need to be sure that a lack of resources is not the cause of false results.

• Multiple operating systems: Many times you will want to test or validate a finding from another system. It is always good to test from different operating systems to see if the results differ.

• Duplicate tools: A great way to validate a finding is to run the same test with a different tool to see if the results are the same.

• Practice targets: You need to practice using your tools. To do this, you need to practice on targets that are known to be vulnerable.

What Tools Should You Use in Your Lab?

Chapter 10 is dedicated to penetration testing tools. Therefore, this section only scratches the surface. Basically, the tools you use in penetration testing depend on the type of testing you are doing. If you are doing testing on a customer environment, you will likely be evaluating various attack surfaces—such as network infrastructure, wireless infrastructure, web servers, database servers, Windows systems, or Linux systems, for example.

Network infrastructure–based tools might include tools for sniffing or manipulating traffic, flooding network devices, and bypassing firewalls and IPSs. For wireless testing purposes, you might use tools for cracking wireless encryption, de-authorizing network devices, and performing on-path attacks (also called man-in-the-middle attacks).

When testing web applications and services, you can find a number of automated tools built specifically for scanning and detecting web vulnerabilities, as well as manual testing tools such as interception proxies. Some of these same tools can be used to test for database vulnerabilities (such as SQL injection vulnerabilities).

For testing the server and client platforms in an environment, you can use a number of automated vulnerability scanning tools to identify things such as outdated software and misconfigurations. With a lot of development targeting mobile platforms, there is an increasing need for testing these applications and the servers that support them. For such testing, you need another set of tools specific to testing mobile applications and the back-end APIs that they typically communicate with. And you should not forget about fuzzing tools, which are normally used for testing the robustness of protocols.

What If You Break Something?

Being able to recover your lab environment is important for many reasons. As discussed earlier, when doing penetration testing, you will break things; sometimes when you break things, they do not recover on their own. For instance, when you are testing web applications, some of the attacks you send will input bogus data into form fields, and that data will likely end up in the database, so your database will be filled with that bogus data. Obviously, in a production environment, this is not a good thing. The data being input can also be of malicious nature, such as scripting and injection attacks. This can cause corruption of the database as well. Of course, you know that this would be an issue in a production environment. It is also an issue in a lab environment if you do not have an easy way to recover. Without a quick recovery method, you would likely be stuck rebuilding your system under test. This can be time-consuming, and if you are doing this for a customer, it can affect your overall timeline.

Using some kind of virtual environment is ideal as it offers snapshot and restore features for the system state. Sometimes this is not possible, though. For example, you may be testing a system that cannot be virtualized. In such a case, having a full backup of the system or environment is required. This way, you can quickly be back up and testing if something gets broken—because it most likely will. After all, you are doing penetration testing.

Exam Preparation Tasks

As mentioned in the section “How to Use This Book” in the Introduction, you have a couple choices for exam preparation: the exercises here, Chapter 11, “Final Preparation,” and the exam simulation questions in the Pearson Test Prep software online.