Remember in Chapter 1, Introduction to Web Application Penetration Testing, that we learned about the penetration testing process. In that process, the second phase was mapping.

In the mapping phase, we need to build a map or catalog of the application resources and functionalities. As a security tester, we aim to identify all the components and entry points in the app. The main components that we are interested in are the resources that take parameters as input, the forms, and the directories.

The mapping is mainly performed with a crawler. Crawlers are also known as spiders, and usually, they perform scraping tasks, which means that they will also extract interesting data from the application such as emails, forms, comments, hidden fields, and more.

In order to perform application mapping, we have the following options:

• The first technique is crawling. The idea is to request the first page, pass all the content, extract all the links in scope, and repeat this with the links that have been discovered until the entire application is covered. Then, we can use an HTTP proxy to identify all the resources and links that may be missed by a crawler. Basically, most of the URLs that are generated dynamically in the browser with JavaScript will be missed by the crawler, as the crawler does not interpret JS.
• Another technique is to discover resources that are not linked anywhere in the application by using dictionary attacks. We'll build our own BruteForcer in the next section.

Here, we have an example of how the Burp proxy creates application mapping using the proxy and the spider functionalities:

We can see the directories, the static pages, and the pages that accept parameters, with the different parameters and the different values.

All the interesting parts will be used for handling vulnerabilities using different techniques such as SQL injection, cross-site scripting, XML injection, and LDAP injection. Basically, the aim of mapping is to cover all the applications in order to identify the interesting resources for the vulnerability identification phase.

In the next section, we'll start developing our own crawler. Let's get ready!