In this section, we will understand what web application penetration testing is and the process behind it. We will start by learning what web application penetration testing is, the importance of performing these tests, what professional methodologies look like, and we'll briefly explain why it is important to have skills to use Python to write our own tools.

Penetration testing is a type of security testing that evaluates the security of an application from the perspective of an attacker. It is an offensive exercise where you have to think like an attacker and understand the developers as well as the technology involved in order to unveil all the flaws.

The goal is to identify all the flaws and demonstrate how they can be exploited by an attacker, and what the impact will be on our company. Finally, the report will provide solutions to fix the issues that have been detected. It's a manual and dynamic test. Manual means that it heavily depends on the knowledge of the person doing the test, and that is why learning how to write your own penetration testing tools is important, and will give you an edge in your career. Dynamic testing is where we test the running application. It is not a static analysis of the source code. The security test is useful to validate and verify the effect of the application security controls to us and to identify the lax of these security controls.

So, why should we perform penetration testing? Nowadays, IT has taken the world by storm. Most of the company processes and data are handled by computers. This is the reason why companies need to invest in security testing, in order to validate the effectiveness of security controls, and many a times the lack of them.

One report by EMC (https://www.scmagazine.com/study-it-leaders-count-the-cost-of-breaches-data-loss-and-downtime/article/542793/) states that the average report regarding annual financial loss per company is 497,037 USD for down time, 860,273 USD for security breaches, and 585,892 USD for data loss. Plus, all the time, the company resources are put into incident response and fixing, testing, and deploying the issue:

That is why performing penetration testing will help companies to protect their customer's data, intellectual property, and services. Penetration testing is a simple methodology formed by four main sections, which are as follows:

• Reconnaissance: In this phase, we'll gather information to identify the technologies used, the infrastructure supporting the application, software configuration, load balances, and so on. This phase is also known as fingerprinting.

• Mapping: We then move into the mapping phase, where we build a map or diagram of the application pages and functionalities. We aim to identify the components and their relationships. One of the techniques to support mapping is spidering or crawling. Also, in this phase, we'll discover nonlinked resources by performing brute force attacks.
• Vulnerability: Once we have all the components, parameters, forms, and functionalities mapped out, we move to phase three, where we'll start vulnerability discovery.
• Exploitation: After identifying all the vulnerabilities, we can move to the last phase, which is the exploitation of the vulnerabilities. Depending on the scope of the pen test, once you exploit vulnerability, you can start the process all over again from your new vantage point. Usually, this the target DMZ, which you would try to get into their internal network segment.

One step that is not represented here is the reporting phase, where you document all the findings so that you can present them to your customer, company.

Finally, there are two types of penetration tests, which are the black box and the white box. Black box test takes place when you don't have any information about the target, which is basically the same situation as an attacker, and white box test takes place when the customer provides us with documentation, source code, and configurations to accelerate the process, and we only focus on interesting areas.

You maybe wondering, what areas should you test during this process? These are some of the most important ones to cover:

• Configuration and deployment management testing
• Identity management testing
• Authentication testing
• Authorization testing
• Session management testing
• Input validation
• Testing error handling
• Cryptography
• Business logic testing
• Client-side testing

We'll cover some of these areas in this chapter.

So, why build your own tools? Web applications are very different since they're developed using multiple technologies, combinations, flows, and implementations.

This is the reason why there is not a single tool that will cover all the scenarios that you will find during your career. Many times, we'll write scripts to test specific issues or to make certain tasks, and to exploit a vulnerability. During the course of this book, we'll see how to write tools and test different areas such as authentication, input validation, and discovery, and we'll end up writing a simple Hypertext Transfer Protocol (HTTP) proxy that could be the foundation of our own security scanner. Writing your own tools is a valuable skill that will put you ahead of many penetration testers that do not have the capability to adapt tools, or write their own. In certain penetration test engagements, this could make all the difference.