Exploring the Commercial Tools

Multiple commercial tools are available on the market today. Purchasing one of these tools can be a daunting and confusing task. As with most products, each vendor’s marketing team will tell you that their product is the best and that they have the most checks. The problem when purchasing such a tool is that not all the vendors count their checks in the same way. Mitre, a U.S. federally funded research and development organization (www.mitre.org) has partially addressed this problem by creating the Common Vulnerabilities and Exposures (CVE) dictionary, which is a standardized naming convention for vulnerabilities and information security exposures. The goal of CVE was to make it easier for both security vendors and the end users to map vulnerability information across the multiple tools. Currently, a number of commercial and freeware products have mapped or are in the process of mapping their databases to CVE numbers. That being said, it is important when evaluating these tools for your own use that you take the marketing numbers with a grain of salt and actually install and run each product before deciding on a purchase. See Table 17.1 for a table of products and their vulnerability count.

As you can see, when based purely on the numbers, each scanner appears to be dramatically different. An ideal solution to this confusion would be if each vendor mapped and counted their checks based on what CVE entry it scans for. This is no small task, and in the case of most vendors, would require not only rethinking how they count checks, but also how various checks are written. As vendors find new ways to show that their product is superior, the checks game will cease to exist and true comparative issues like false positive rate, scan engine performance, usability, and reporting features will become the key indicators as to which product is superior.

Here’s a quick review some of the criteria that you should consider when purchasing a commercial scanning product:

You need to understand that most commercially vulnerability scanners are not created equal, and each has its own strengths and weaknesses. It is common to find security administrators using more than one commercial tool, because no one product is a complete fit for every network. When deciding on a vulnerability scanner, you need to take the time to thoroughly evaluate each product for your specific needs and environment. Almost all product vendors will offer you a free demonstration copy of their software—take them up on this offer. The worst-case scenario is that you will find yourself being phoned by their sales people to assist you in making a decision. If the salesperson cannot answer your questions sufficiently, ask to speak to one of the product engineers. My experience with vendors has usually been good as they are happy to help and answer any of your questions, but be wary of the marketingspeak. Make your own decision as to what product will fit your needs.

False positive rates are probably the most annoying issue you will have with vulnerability scanners. A false positive is when the scanner reports that an issue exists when it really does not. A high rate of these will cause you to stop trusting the scanner and start verifying, usually manually, each find. Obviously, this isn’t productive and would make you wonder why you purchased an expensive scanner in the first place. False negatives—when the scanner does not detect an issue that does in fact exist—are even more disturbing. Luckily, these are less common and easier for a vendor to fix, but have been known to exist. This alone is probably the best reason to use more than one scanner, and of course, constant monitoring of your systems.

If you are responsible for a large network, scanner performance is probably important to you. A lot of factors affect the performance of the product. Two of the more obvious factors are the scanner engine itself and how the vendor has decided to check for the existence of a vulnerability. Today, most products are multithreaded applications that allow for a bit of user tuning. The bottom line when comparing scanner performance is that when you are scanning multiple machines, you can only do so much to tune performance. Some vendors have addressed this problem by offering distributed scanning solutions that use multiple scan engines on multiple machines to scan the network then report back to a central reporting console. In theory, this sounds like an acceptable solution, but it opens the floor to other issues, such as network bandwidth, and, of course, the potential security issues if the traffic isn’t handled securely.

Reporting is a feature that is slowly becoming standardized among all the scanning products on the market. Whether the product uses its own custom reporting solution or has Crystal Report functionality built in, most of them allow the user to customize the report output.

Figure 17.1 shows the interface for one common commercial scanner, ISS Internet Scanner, and Figure 17.2 shows the interface for another, Retina by eEye. As you can see, the interfaces do have their subtle differences, but both are intuitive and easy to use. You will not find a large difference between the usability of each of the established commercial products, but as you will see later in this chapter, you do have to be aware of and understand their limitations.

We don’t write a lot about each commercial product—the links in the “Automated Tools: Product Reviews” sidebar all lead to specific product reviews—but we do list of some of the common ones and give a short blurb about each product based on our own experiences with them.

Exploring the Free Tools

Everybody likes getting something for free. The general rule however, has always been “you get what you pay for.” I would argue that in the case of vulnerability scanners, the general rule is actually the exception. One caveat though, you need to understand the limitations and expectation of freeware and open source software.

These are not packages that have large development teams who get paid for their work; they are packages that are developed by intelligent people in their spare time. Support is typically sparse, and operating most of these tools is not as easy as clicking on an icon. That being said, the freeware and open-source tools have their place and most of them do the job as advertised.

This section takes a look at some of the popular tools (Nessus, SAINT, SARA, ShadowScan, Nmap, whisker, and VLAD), what they do, and how effective they are. Of course, your experience with each tool may differ from ours, but we try to present all of the issues—good and bad.

Testing with the Commercial Tools

Let’s look at the original scenario where you have to perform a penetration test on five systems with the IP addresses 192.168.0.1 through 192.168.0.5. This is all of the information you have been provided, no operating system information and no listening services information. How can a commercial automated tool help you make this process as efficient as possible? First, you need to purchases a license for the selected tool. Whether you choose ISS Internet Scanner, Network Associates CyberCop, or eEye Retina, the process from here is very similar.

Simply launch the tool, give it the necessary information, then enter in the IP address range you wish to scan. Some commercial tools give you the ability to preselect the type of scan you wish to perform, as shown in Figure 17.4, which is the scan policy selection screen from ISS Internet Scanner.

From this point, you need to simply wait until the scan completes then analyze the results and create a report. The next steps from here vary. Unfortunately, a large population of consultants and consulting organizations think that the next logical step from here is to hand over the report and attach an invoice.

What should be done, instead of simply handing over the report, is that you should analyze the report results and, where necessary, manually verify the results. The commercial tool is great to determine a baseline in which you should now base some real work. For example, say that the commercial tool claimed to find all five hosts vulnerable to the Windows NT Internet Information Server showcode.asp vulnerability. A wise move would be to manually test each system to verify that they are truly vulnerable. First, you need to first verify that each system is actually a Windows NT system running Internet Information Server. You can accomplish this in a couple of different ways (probably more); the first is by using the Telnet command as follows:

telnet www.example.com 80

HEAD / HTTP/1.0<enter><enter>

HTTP/1.1 200 OK

Server: Microsoft-IIS/5.0

Date: Mon, 04 Feb 2002 21:41:17 GMT

Connection: Keep-Alive

Content-Length:  19398

Content-Type: text/html

Cache-control: private

As you can see, the information returned identified the system as a Microsoft IIS 4.0. Another way that you can identify the operating system running on the host is to simply go to Netcraft at http://uptime.netcraft.com/up/graph/ and enter the IP address or URL of the site in question. Figure 17.5 shows the output from Netcraft. As you can see, Netcraft identifies the remote operating system and provides potentially valuable uptime information.

Pretend that you decided to use the Telnet method on all five hosts. On the last host tested, you receive the following information:

telnet www.example.com 80

HEAD/HTTP/1.0<enter><enter>

HTTP/1.1 200 OK

Date: Mon, 04 Feb 2002 21:48:31 GMT

Server: Apache/1.3.19 (Unix) mod_ssl/2.8.4 OpenSSL/0.9.6b

Last-Modified: Tue, 29 Jan 2002 15:13:47 GMT

ETag: “21-1a7a-3c56bc2b”

Accept-Ranges: bytes

Content-Length: 6778

Connection: close

Content-Type: text/html

The final system seems to be Apache running on a UNIX variant. Now you know that the IIS vulnerability is incorrect on the fifth host, and you do not need to test it further, and that this host might perhaps be vulnerable to an Apache or UNIX vulnerability and should be further investigated later. This is also a large clue that perhaps the other four systems, although they certainly seem to be running Windows NT with Microsoft IIS 5.0 Server, should be further tested to ensure that the vulnerability actually exists.

To accomplish this test, you need to have knowledge of the vulnerability. Unfortunately, the commercial tools do not help much here—some of them will give pointers on the Internet where you can go and read about the vulnerabilities. Fortunately, the Internet has multiple resources that catalog vulnerability information, complete with how to “test” for such a vulnerability. One such resource is www.securityfocus.com. By doing a search at securityfocus.com for “showcode.asp”, you can find the URL www.securityfocus.com/bid/167, which provides you with all the information you need. Using your Web browser, you can types in the following URL: www.example.com/msadc/Samples/SELECTOR/showcode.asp?source=/msadc/Samples/../../../../../boot.ini

In the browser window, you should now see the contents of the BOOT.INI file located in the root of all Windows NT installations. If the file is not displayed, you should attempt the same exploit using other known, readable files. Once the vulnerability has been adequately tested, you can determine if the hosts are truly vulnerable by your ability to view readable files. Screenshots of these readable files also make great report additions to further drive the point home.

As you can see, using the commercial scanning tools help make testing hosts for vulnerability much more efficient. Imagine attempting to test these hosts without an automated tool; the current CVE database is at 1,604 entries (as of January 13, 2002), which makes trying to manually test for every applicable vulnerability a daunting task. With the assistance of an automated tool, you simply need to verify the results and retest any systems that return enough anomalies to cause you to not trust the scanner. These anomalies, and the prospect of having to completely manually test a host, are what cause many consultants to use more than one scanning product—typically they will use a commercial tool and a freeware tool.

Testing the Free Tools

Like the preceding scenario with commercial tools, you can also use free tools in the same manner. Free tools are probably more accurate because they require a little more user input and interaction. Let’s describe two separate scenarios with the same five hosts. The first scenario will describe a situation where you need to rely on multiple free tools and your own knowledge to test the systems. Before getting into the example, we want to make one thing clear: We know that there are multiple ways to do what we are describing, there are probably even more efficient ways than we are describing. We simply using some common examples to help illustrate a point.

First, you can uses Nmap to scan the five hosts and determine what ports are open by using the following syntax:

nmap –sS –v –v –0 –P0 –oN results.out 192.168.0.1-5

Nmap will then scan all five systems and return information that should look something like this:

According to the output of this scan, the host at 192.168.0.1 is running NT Server 4.0 and has a Web server installed that is listening on ports 80 (http) and 443 (https). It would probably be a good idea to now confirm that the Web server running is IIS by either using Netcraft or Telnet as explained in “Testing with the Commercial Tools.” Once you confirm, you have a number of options at your disposal. The first being to manually go through and test each related IIS vulnerability, which, of course, might be a bit too time consuming. The second would be to use either Whisker or VLAD, to quickly check for some of the more common IIS vulnerabilities, and as you learned from using the commercial tool on this host, the showcode.asp vulnerability.

Obviously, the Nmap method shown, while probably more precise, does leave room for error and room for missing vulnerabilities. Typically, you would use this method to go after the “low-hanging fruit,” or common vulnerabilities. Also, instead of using VLAD or Whisker to test the Web server, it would be a simple task to create a Perl script that quickly scans a Web server for most of the common IIS vulnerabilities, such as double decode, unicode, and any of the sample pages exploits, such as showcode.asp.

A second option to test these five systems is to use one of the freeware security scanners, such as SAINT, SARA, or Nessus. In my opinion, SAINT and SARA do not provide an in-depth enough scan to be effective in this case, so by default, use Nessus, which is probably the best freeware scanner available.

Nessus works in a manner very similar to the commercial scanning products. Once connected to the Nessus server, you can log in and select what options you want to scan for, as shown in Figure 17.6. Additionally, you can also set what type of portscan you would like Nessus to perform, as shown in Figure 17.7. As you can see in both of these screen shots Nessus removes the need to first run Nmap then run a custom script as all of the options you need are built right in.

Like the commercial scanners, however, Nessus can be prone to the occasional false positive or incorrectly identified host. So, as with the commercial tools, performing some sort of sanity checking on the reports and verifying information as required would be wise.

The New Face of Vulnerability Testing

During July 2001, at The Black Hat Briefings in Las Vegas, NV, Ivan Acre, and Máximiliano Cáceres of CORE-SDI, presented their work in the area of penetration testing and automated penetration testing. Their theory is that the current methodologies used to perform penetration testing are not as effective or optimal as they could be. Additionally, the typical automated scanning tool will scan a host, identify vulnerabilities, and not actually break into the host being scanned or attempt to look at any other hosts that might be connected in some way.

CORE-SDI has done a considerable amount of work in developing new tools to help automate the entire penetration testing process from the initial information gathering phase to the actual exploitation of the hosts. Some of the key benefits of this approach would be a tool that encompasses the entire penetration test under one common framework, to define and enforce a standardized methodology, to improve on the security of the penetration tests, and finally, to accurately speed up monotonous and time-consuming tasks.

I personally feel that CORE-SDI has the potential to revolutionize the penetration testing field and raise the bar on vulnerability scanning. Quite some time has passed since the presentation at Black Hat, but rumor has it that CORE is close to releasing beta versions of their tool. As someone who performs a lot of penetration tests every year, I look forward to seeing what CORE-SDI has to offer because it should not only improve on the quality of work presented by penetration testers but also increase the value of a penetration test to organizations while making it more cost effective.