Web application assessments, or web application penetration tests, are a different animal compared to infrastructure assessments. This is dependent on the goals of the assessment as well. Web application assessments, like mobile application assessments, are all too often approached in the wrong manner. Network or infrastructure penetration tests have matured, and clients are becoming wiser in what to expect for results. This is not always true for web application or mobile application assessments. There are a variety of tools that can be used to analyze applications for vulnerabilities, including Metasploit, Nexpose, Nessus, Core Impact, WebInspect, AppScan, Acunetix, and many more. Some are far better than others for web application vulnerability assessments, but they all have a few things in common. One of these things is that they are not a replacement for penetration tests.
These tools have their place, but depending on the scoping of the engagement and what weaknesses are trying to be identified, they often fall short. Specific products such as WebInspect, AppScan, and Acunetix are appropriate for identifying potential vulnerabilities, especially during the System Development Life Cycle (SDLC), but they will report false positives and miss complex multistage exploits. Every tool has its place, but even when using tools such as these, relevant risks can be missed.
Now there is a flip side to this coin; a penetration test will not find every vulnerability in a web application, but it is not meant to do so anyway. Web application penetration tests are focused on identifying systematic developmental problems, processes, and critical risks. So, the identified vulnerabilities can be quickly remediated, but the specific weaknesses point to larger security practices that should be addressed in the overall SDLC.
The focus of most application penetration tests should involve at least some components out of the following, if not all:
- Analysis of the current Open Web Application Security Project (OWASP) top 10 vulnerabilities.
- Identification of application areas that leak data or leave residual data traces in some locations, which includes undocumented or unlinked pages or directories. This is also known as data permanency.
- Manners in which a malicious actor could move laterally from one account type to another or escalate privileges.
- Areas in which the application could provide an attacker with the means to inject or manipulate data.
- Ways in which the application could create Denial of Service (DoS) situations, but this is typically accomplished without exploitation or explicit validation to prevent any impact on business operations.
- Finally, how an attacker could penetrate the internal network.
Consider all of these components and you will see that the use of an application scanning tool will not identify all of them. Additionally, a penetration test should have specific objectives and goals to identify indicators and issues with relevant proof of concepts. Otherwise, if an assessor attempts to identify all the vulnerabilities in the application depending on complexity, it could take an extensive period of time.
These recommendations and the application code should be reviewed by the client. The client should remediate all the specified locations highlighted by the assessor and then follow through and identify other weaknesses the assessor may not have identified during the time period. Once completed the SDLC should be updated so that future weaknesses are remediated in development. Finally, the more complex the application, the more the developers involved; so as you test it, be aware of vulnerability heat mapping.
Just like penetration testers, developers can have varied levels of skills, and if the organization's SDLC is not very mature, the grade of vulnerability in the application areas can vary for each development team. We call this vulnerability heat mapping, where some places in an application we will have more vulnerabilities than others. This typically means that the developer, or developers, did not have the necessary skills to deliver the product at the same level as the other teams. Areas where there are more vulnerabilities may also indicate that there are more critical vulnerabilities. So, if you notice that a specific area of an application is lighting up like a Christmas tree with weaknesses, elevate the type of attack vectors you are looking at.
Depending on the scope of the engagement, start focusing on vulnerabilities that will crack the security perimeter, such as Structured Query Language injection (SQLi), Remote or Local File Inclusion (RFI/LFI), nonvalidated redirects and forwards, unrestricted file uploads, and finally insecure direct object references. Each of these vulnerabilities are related to the manipulation of the request-and-response model of the application.
Applications typically work on a request-and-response model, with tracking of specific user session data with cookies. Therefore, when you write your scripts, you have to build them in a method to handle sending data, receiving it, and parsing the results for what was expected or not expected. Then, you can create follow-on requests to move further ahead.
When assessing large environments to include Content Delivery Networks (CDN), you will find that you will be identifying hundreds of open web ports. Most of these web ports have no active web applications deployed on those ports, so you need to either visit each page or request the web page header. This can simply be done by executing a HEAD request to both the http:// and https:// versions of the site. A Python script that uses urllib2 can execute this very easily. This script simply takes a file of the host Internet Protocol (IP) addresses, which then builds the strings that create the relevant Uniform Resource Locator (URL). As each site is requested, if it receives a successful request, the data is written to a file:
#!/usr/bin/env python
import urllib2, argparse, sys
defhost_test(filename):
file = "headrequests.log"
bufsize = 0
e = open(file, 'a', bufsize)
print("[*] Reading file %s") % (file)
with open(filename) as f:
hostlist = f.readlines()
for host in hostlist:
print("[*] Testing %s") % (str(host))
target = "http://" + host
target_secure = "https://" + host
try:
request = urllib2.Request(target)
request.get_method = lambda : 'HEAD'
response = urllib2.urlopen(request)
except:
print("[-] No web server at %s") % (str(target))
response = None
if response != None:
print("[*] Response from %s") % (str(target))
print(response.info())
details = response.info()
e.write(str(details))
try:
response_secure = urllib2.urlopen(request_secure)
request_secure.get_method = lambda : 'HEAD'
response_secure = urllib2.urlopen(request_secure)
except:
print("[-] No web server at %s") % (str(target_secure))
response_secure = None
if response_secure != None:
print("[*] Response from %s") % (str(target_secure))
print(response_secure.info())
details = response_secure.info()
e.write(str(details))
e.close()The following screenshot shows the output of this script on the screen as it is run:
Note
The full version of this script can be found at https://raw.githubusercontent.com/funkandwagnalls/pythonpentest/master/headrequest.py. This script can easily be modified so as to execute follow-on tasks, if desired. There are already tools such as PeppingTom and EyeWitness available that accomplish this activity better than this script, but understanding how to build this basic script will allow you to include additional analysis as necessary.