IDS Evasion
Summary
Solutions Fast Track
Frequently Asked QuestionsIntroduction
One of the laws of security is that all signature-based detection mechanisms can be bypassed. This is as true for intrusion detection system (IDS) signatures as it is for virus signatures. IDS systems, which have all the problems of a virus scanner, plus the job of modeling network state, must operate at several layers simultaneously, and they can be fooled at each of those layers.
This chapter covers techniques for evading IDSs, which include playing games at the packet level, application level, and morphing the machine code. Each of these types can be used individually, or together, to evade detection by an IDS.
In this chapter, we present several examples of how an attack might evade detection.
Understanding How Signature-Based IDSs Work
An IDS is quite simply the high-tech equivalent of a burglar alarm—a burglar alarm configured to monitor access points, hostile activities, and known intruders. These systems typically trigger on events by referencing network activity against an attack signature database. If a match is made, an alert will take place and will be logged for future reference. It is the makeup of this signature database that is the Achilles heel of these systems.
Attack signatures consist of several components used to uniquely describe an attack. An ideal signature would be one that is specific to the attack while being as simple as possible to match with the input data stream (large complex signatures may pose a serious processing burden). Just as there are varying types of attacks, there must be varying types of signatures. Some signatures will define the characteristics of a single Internet Protocol (IP) option, perhaps that of an nmap portscan, while others will be derived from the actual payload of an attack.
Most signatures are constructed by running a known exploit several times, monitoring the data as it appears on the network and looking for a unique pattern that is repeated on every execution. This method works fairly well at ensuring that the signature will consistently match an attempt by that particular exploit. Although I have seen my share of shoddy signatures, some so simplistic in nature that the amazingly hostile activity of browsing a few Web sites may set them off, remember the idea is for the unique identification of an attack, not merely the detection of attacks.
The following are example Snort signatures:
Snort implements a description language used to construct any rule. To avoid getting into the rather complex details of writing your own signatures, let’s simply go left to right through the preceding examples and try to discern what exactly they mean. We can see that these all define a type of alert. These alerts are then classified into a type of protocol where the specific details are given: IP address ($EXTERNAL_NET and $HOME_NET are variables usually defined as 10.10.10.0/24 CIDR style) and port numbers to restrict the scope. The msg keyword defines the message that will be sent out if the rule is matched; flags will define which of the Transmission Control Protocol (TCP) flags are set in the stream; ipopts dictates the options of an IP packet; and content is used to specify a unique series of data that appears in the actual contents of the packet. In a content field, anything between vertical bars is in hex format, while the rest is ASCII.
The first rule watches for any attempt from the outside to connect to an inside host at TCP port 8080, which is a port often used for Web proxies. The second rule looks for a commonly-used shellcode sequence inside any IP packet going to a port less than 1024. (The :1023 is shorthand for a range of ports between 0 and 1023, inclusive.) The third rule is checking for a CWD … command to TCP port 21, the File Transfer Protocol (FTP) port. The fourth rule is watching for IP packets with the Record Route (rr) option on. The final rule is checking for the string fusr/bin/chgrp going to port 80, the Hypertext Transfer Protocol (HTTP) port.
Computing systems, in their most basic abstraction, can be defined as a finite state machine, which literally means that there are only a specific predefined number of states that a system may attain. This limitation hinders the IDS in that it can be well armed at only a single point in time (in other words, as well armed as the size of its database). First, how can one have foreknowledge of the internal characteristics that make up an intrusion attempt that has not yet occurred? You can’t alert on attacks you’ve never seen before. Second, there can be only educated guesses that what has happened in the past may again transpire in the future. You can create a signature for a past attack after the fact, but that’s no guarantee you’ll ever see that attack again. Third, an IDS may be incapable of discerning a new attack from the background white noise of any network. The network utilization may be too high, or many false positives cause rules to be disabled. And finally, it may be incapacitated by even the slightest modification to a known attack. It is either a weakness in the signature matching process, or more fundamentally, a weakness in the packet analysis engine (packet sniffing/reconstruction) that will thwart any detection capability.
The goals of an attacker as it relates to IDS evasion are twofold: To evade detection completely, or to use techniques and methods that will increase the processing load of the IDS sensor significantly. The more methods employed by attackers at large, on a wide scale, the more vendors will be forced to implement more complex signature matching and packet analysis engines. These complex systems will undoubtedly have lower operating throughputs and more opportunities for evasion. The paradox is that the more complex a system becomes, the more opportunities there are for vulnerabilities! Some say the ratio for bugs to code may be as high as 1:1000, and even conservatives say a ratio of 1:10000 may exist. With these sorts of figures in mind, a system of increasing complexity will undoubtedly lead to new levels of increased insecurity.
Judging False Positives and Negatives
To be an effective tool, an IDS must be able to digest and report information efficiently. A false positive is a triggered event that did not actually occur, which may be as innocuous as the download of a signature database (downloading of an IDS signature database may trigger every alarm in the book) or some unusual traffic generated by a networked game. This, although annoying, is usually not of much consequence, but can easily happen and is usually tuned down by an initial configuration and burn-in of a network IDS (NIDS) configuration. More dangerous, however, is the possibility for false negatives, which is the failure to be alerted to an actual event. This would occur in a failure of one of the key functional units of a NIDS. False negatives are the product of a situation in which an attacker modifies their attack payload in order to subvert the detection engine. False positives have a significant impact on the effectiveness of an IDS sensor. If you are charged with the responsibility of monitoring a device, you will find you become accustomed to its typical behavior. If there is a reasonable number of false positives being detected, the perceived urgency of an alert may be diminished by the fact that there are numerous events being triggered on a daily basis that turn into wild goose chases. In the end, all the power of IDS is ultimately controlled by a single judgment call on whether or not to take action.
Alert Flooding
This problem of making sense of what an IDS reports is apparent again in a flood scenario. Flooding, as you may have guessed, is the process of overloading the IDS by triggering a deluge of alerts. This attack has a number of beneficial actions for the perpetrator. If the attacker can muster enough firepower in terms of network bandwidth, a denial of service (DoS) attack is possible.
Many IDS sensors exasperate this condition by the first match (or multiple match) paradox, in which the sensor has to essentially decide whether or not to alert based on the first match in its database or to attempt further matches. The issue here is that an attacker may identify a low-priority or benign signature common to many IDS signature databases and attempt to reproduce this in a more damaging exploit attempt. If the sensor were to use a first match method, it would produce an alert for the less severe vulnerability and not signal the true nature of the attack. However, in using the multiple match approach, the IDS allows itself to be more vulnerable to alert flooding attacks. The attacker may simply package an entire signature database into some network traffic and watch the IDS crumble to the ground.
Aside from the desirable condition of failing an IDS sensor, there is the added bonus of having generated an excessive amount of alerts (in excess of 10,000 is no problem at all) that the admin must then somehow make sense of. The intended target host may be totally lost within a dizzying display of messages, beeps, and red flags. Trying to identify a real intrusion event may be arduous at best. Let us not forget the psychological impact of seeing what may be construed as an all-out Internet-wide assault on your networking equipment. If this style of attack were to somehow become routine, how effective would your IDS solution be then?
Using Packet Level Evasion
Network IDSs have the dubious task of making sense of literally millions of pieces of information per second, analyzing information while providing acceptable response times (typically as close to real-time as possible is desired). To break down the effort of data analysis, a NIDS will function on several discrete layers of the network protocol stack. The first layers under inspection will be the network and transport layers, where the attacker has a great opportunity to confuse, circumvent, or eliminate a NIDS sensor. If an attacker were to devise a technique that would enable them to evade detection, this would be an ideal location to begin, as all other detection capabilities of the IDS rely on the ability to correctly interpret network traffic just as the target host would.
Unfortunately for the defender, the characteristics of IP and TCP do not lend themselves to well-defined inspection. These protocols were developed to operate in a dynamic environment, defined by permissive standards that are laden with soft “SHOULD” and “MAY” statements, “MUST” being reserved for only the most basic requirements. This lax definition of protocol standards leads to many complications when an attempt is made to interpret network communications. This will leave the door open for an attacker to desynchronize the state of the IDS, such that it does not correctly assemble traffic in the same manner that the target host will. For example, if an IDS signature was crafted to search for the string “CODE-RED” in any HTTP request, it may be possible for the attacker to fragment his traffic in such a way that it will assemble differently for the IDS than for the target host. Therefore, the attacker may exploit the target host without the IDS being able to interpret the event accordingly.
Several years ago, a paper was written to discuss the many issues facing NIDS development. Essentially, the attacks discussed in Thomas Ptacek and Timothy Newsham’s 1998 “Insertion, Evasion, and Denial of Service: Eluding Network Intrusion Detection,” (http://secinf.net/info/ids/idspaper/idspaper.html), vary in style from insertion to evasion attacks. Insertion and evasion are the basis for evading a signature match.
Insertion is the technique which relies upon a situation in which an IDS will accept some information with the assumption that the target host will also. However, if the IDS does not interpret the network stream in the same manner that the target does, the IDS will have a different understanding of what the communication looks like and will be ineffective in properly alerting to the presence of an attack. The IDS signature will simply not match the data acquired from the network. Our “CODE-RED” example may be seen to the IDS as “CODE-NOT-RED”, which may be enough for the IDS to feel safe, whereas the target host will actually receive “CODE-RED”, having dropped the “NOT” in the middle due to the packet containing it not matching the target’s understanding of the standards.
Evasion is the converse of insertion; it relies upon a situation in which a target system will accept data that the IDS will ignore. An attack may then look something like “CODE” to the IDS where the target will receive “CODE-RED”. These sorts of attacks can be enabled in a number of ways. At any time a TCP/IP communication may be terminated by either party. If the IDS were to incorrectly interpret a RST or FIN from an attacker that was not accepted by the target host (for example, if the IDS did not correctly monitor sequence numbers), the attacker would be free to communicate with impudence.
Denial of service in IDS implementations is commonplace. The opportunities to subvert the operation of a sensor are quite apparent. System resources are finite; there are only so many pages of memory that can be allocated; CPUs are bound and even network IO cards may not be able to maintain consistent throughput despite their speed rating. Because a computer is a system of queues, some will inevitably fill and spill faster than the data contained may be examined. These issues vary from the micro scale, when we are concerned with exhausting the relatively few network IO buffers, to macro issues similar to running low on disk resources. Management of system resources is a complex task that is made exceedingly difficult by requirements to monitor an unknown amount of communication streams and a limited view of the actual internal TCP/IP stack state for each host.
IP Options
Upon examination of an IP header, there are a number of fields in which, with methodical alteration, some insertion or evasion vulnerabilities will become apparent. Mangling the IP header must be done with care; our traffic must still be valid such that it can be routed across the Internet. Modifying the size of a packet may make it difficult for the IDS to understand where the upper layers of the packet begin (evasion). The IP checksum is another good starting point. If we can interleave invalid IP packets in our stream, the IDS may accept them as valid (if it does not manually calculate the checksum for every packet) where the end system does not (insertion).
Time-To-Live Attacks
In a typical network configuration, the NIDS would most often be placed on the perimeter of a network. This would enable the NIDS to monitor all communication across the Internet. Unfortunately, if an attacker is able to traceroute or methodically reduce the Time to Live (TTL) of the traffic to the target and identify the exact amount of hops required to reach the host, they would then be able to send some packets with an insufficient TTL value. This would have the effect of ensuring the packets with a lower TTL would never reach the target system, but would instead be possessed by the IDS as part of the stream, as seen in Figure 16.1. Luckily administrators may be able to combat this attack by configuring their IDS on the same network segment as the hosts they wish to monitor.
IP Fragmentation
IP fragmentation reassembly is the basis for a number of attacks. If the NIDS sensor does not reassemble IP fragments in a similar fashion as the target host, it will not be able to match the packet to its signature database. In normal network operations, IP fragments will typically arrive in the order in which they are sent. However, this is not always the case; IP supports difficult-to-analyze out-of-order transmission and overlapping fragment reassembly behaviors. Assembling IP fragments can also become complicated by the requirement to keep fragments in memory until the final fragment is received, in order to complete the assembly of the entire packet. This raises yet another DoS issue—many fragments can be transmitted to consume any internal buffers or structures so that the IDS may begin to drop packets or even crash.
We can further elaborate on this issue when we add the complexity of internal garbage collection. An IDS listening to the wire may have to account for the sessions of several thousand hosts, whereas each host need only be concerned with its own traffic. A host system may allow an excessive amount of time for fragments to arrive in the stream, whereas the IDS may have more aggressive timeouts in order to support the management of an exponentially larger system. If the perpetrator were to send an attack consisting of three fragments and withhold the final fragment until a significant amount of time has expired, and if the NIDS does not have identical internal fragment management processes (something tells me this is next to impossible to attain), it will not have a consistent view of the IP packet and will therefore be incapacitated from any signature matching processes.
Fragmentation Tests conducted by Ptacek and Newsham revealed that at the time of testing none of the IDS platforms analyzed could properly interpret a number of IP fragmentation issues. The first two tests covered involved an in-order fragmented payload that was sent in two different sizes (8 and 24 bytes). Further testing was done where 8-byte fragments were sent—with one fragment sent out of order (evasion), with a fragment sent twice (insertion), with all fragments out of order and one duplicate (combination), by sending the fragment marked as the last fragment first (evasion), and by sending a series of fragments that would overlap the previous (evasion). Startling as it may seem, none of the four products (RealSecure, NetRanger, SessionWall, and NFR) were able to handle any of the fragmentation attacks.
Currently, most NIDSs have updated their fragmentation assembly engines such that they are capable of reconstructing streams with some degree of success.
TCP Header
The TCP header contains a number of fields open to exploitation, and so opportunities for evasion and insertion exist if an IDS does not fully inspect the TCP header. The CODE field defines the type of message being sent for the connection; if someone were to send an invalid combination or a packet missing the ACK flag, it would be possible that the target host would reject the packet where the IDS would not (insertion possible). Segments marked as a SYN may also include data; due to the relative infrequent use of this option for data, an IDS may ignore the contents of these types as well (evasion). We can examine many of the fields in the TCP header and look for any opportunity where a target host will either accept traffic that the IDS does not, or vice versa. Another great example is the “Checksum” field, where if the IDS were not manually calculating the checksum for every TCP segment, we may intermix segments with an invalid checksum into our legitimate session with the hope that the IDS will not validate all segments (the vendor may have assumed the processing overhead too great).
TCP recently added several new TCP options with RFC 1323, “TCP Extensions for High Performance,” by V. Jacobson, R. Braden and D. Borman, which introduces (amongst other things) Protection Against Wrapped Sequence (PAWS) numbers and the option for non-SYN packets to contain new option flags. This means that if an IDS does not know how a target system may deal with non-SYN packets containing options, there are multiple opportunities for insertion and evasion. The target system may reject this newer form of TCP where the IDS will not, and again the converse is also true. PAWS is a mechanism where a system will have a timestamp associated with each TCP segment. If the target host were to receive a segment with a timestamp less then its internal threshold value, it will be dropped. Again and again we see the difficulty with examining TCP data on the wire. There is simply not enough state information transmitted to give an accurate picture of what the behavior will be of a potential target host.
TCP Synchronization
Just as there are a number of attack vectors available against strictly IP communications, when we begin to analyze layers above IP, the added complexity and requirements for functionality produce new synchronization challenges. Today most IDS platforms have implemented “stateful” inspection for TCP.
Stateful inspection requires a number of design decisions about how to identify a communication stream when you examine TCP data. An IDS must be capable of reconstructing a stream in an identical manner as the destination host—if it can not, there will be opportunities for an attacker to subvert the analysis engine. The state information for a TCP session is held in a structure known as a TCP Control Block (TCB). A TCB (containing information like source and destination, sequence numbers and current state) will be required for each session that a NIDS will monitor. The three attack vectors that Ptacek and Newsham identified are as follows:
An IDS would have to participate in these processes to identify new sessions, monitor open connections, and to identify when it is appropriate to stop monitoring.
TCB Creation
Understanding how to begin monitoring a connection poses some interesting challenges. Should the NIDS simply monitor the TCP handshake processes and build a TCB at this time? Can the NIDS effectively establish a TCB for a connection for which it did not see a SYN (connections that were active before the monitor)?
There are unique challenges with any technique used to establish a TCB. It would be desirable for the IDS to be able to monitor connections for which it did not see an initial Three-Way Handshake (3WH). If not, an attacker could establish a connection and wait a significant amount of time; the IDS may reboot and then be unable to track the already established connection.
It is possible to only use ACK packets for TCB creation. This is known as synching on data. With the added benefit of being able to identify sessions for which a 3WH has not been inspected. There are a number of drawbacks, one being that the IDS will likely inspect excessive amounts of data as it will not be able to differentiate packets not part of a stream from established connections. Another issue is that syncing on data causes a dependence on accurate sequence number checking. The attacker may be able to desynchronize the IDS by spoofing erroneous data before attempting the attack.
An alternate technique to TCB creation is to require a SYN+ACK combination to be seen. This will have the added benefit that it is nearly impossible for the attacker to effect the ACK from the target network. This will enable the IDS to identify which host is the server and client. However, the IDS may be able to be tricked into tracking many connections for non-existent hosts (DoS). A SYN+ACK can be easily spoofed without requiring the final ACK from the originating host; care should be taken when relying on this mechanism for TCB creation.
A combination of methods is usually the best strategy, building on the strengths while attempting to eliminate the weaknesses of each technique.
Stream Reassembly
A number of similar issues exist for TCP stream reassembly as for IP fragmentation assembly. The TCP segments may arrive out of order, overlap, and possibly be redundant. The IDS must take special care to monitor the sequence numbers of each connection to ensure they do not get desynchronized (difficult to do in a heavily–loaded environment).
Again, the difficulty with interpreting the possible behavior of the destination host, while not knowing the particulars about its TCP/IP stack implementation, is quite challenging. In the case of a redundant TCP segment, some hosts may retain the older frame, while others may discard it in favor of the most recently received.
If an IDS hopes to maintain a consistent view of the traffic being evaluated, it must also be weary of the advertised windows size for each connection; this value is often tuned during a session to ensure maximum throughput. If an IDS were to lose sight of the size of the TCP window, it may be vulnerable to an easy insertion attack where the attacker simply sends in excess of the window size, in which case the destination host will simply drop packets that were received outside of its advertised size.
TCB Teardown
To ensure that a DoS condition does not occur, proper garbage collection must take place. There are some challenges here. Connections may terminate at any time, with or without notice. Some systems may not require RST segments to be properly sequenced. The Internet Control Message Protocol (ICMP) may even terminate a connection; most hosts will respect an ICMP destination unreachable message as an appropriate signal for termination. If the IDS is not aware of these semantics, it may become desynchronized and unable to track new connections with similar parameters.
There will almost undoubtedly be some timeout for any established connection to prevent some logic error from eventually leaking memory. This will also lead to an attack that we had alluded to earlier. Most hosts do not employ keep-alive messages for all connections. This leaves an IDS in an undesirable position where an attacker may simply wait for an excessive amount of time and possibly simultaneously provoke the IDS to become more aggressive with its garbage collection (by establishing many new connections). If successful, the attacker will be able to send whatever attacks they wish, undetected.
Using Fragrouter and Congestant
Theory is not enough for some to make a judgment on the performance of security products. We have seen time and time again that many vendors do not heed the warning of the research community. To adequately illustrate the vulnerabilities that NIDSs face, Dug Song released fragrouter in September 1999 (www.monkey.org/˜dugsong/fragrouter-1.6.tar.gz). Fragrouter’s benefit is that it will enable an attacker to use the same tools and exploits they have always used without modification. Fragrouter functions, as its name suggests, as a sort of fragmenting router. It implements most of the attacks described in the Ptacek and Newsham paper.
Congestant is another great tool that implements a number of anti-IDS packet mangling techniques. This was authored by “horizon” and was first released in December 1998 in his paper, “Defeating Sniffers and Intrusion Detection Systems” (www.phrack.org/show.php?p=54&a=10). The difference here is that Congestant is implemented as a shared library or a kernel patch to OpenBSD. You may find it is possible to use these tools concurrently for some added confusion for the IDS sensor.
Increasing the processing overhead and complexity of IDS sensors is of benefit to an attacker; these systems become more prone to DoS and are less likely to perform in an environment of extreme stress (large numbers of packets per second). It is a certainty that there will always be more features and options added to IDSs as they mature, since an attacker will always attempt to identify the critical execution path (the most CPU intensive operation an IDS may make) in attempts to stress an IDS sensor.
Here is the output when running fragrouter from a shell. It’s pretty plug-and-play, you just need to ensure that your system will route through the “fragrouter” host to reach the target:
storm:˜/dl/fragrouter-1.6# ./fragrouter –F5
fragrouter: frag-5: out of order 8-byte fragments, one duplicate
truncated-tcp 8 (frag 21150:8@0+)
10.10.42.9 > 10.10.42.3: (frag 21150:8@16+)
10.10.42.9 > 10.10.42.3: (frag 21150:8@8+)
10.10.42.9 > 10.10.42.3: (frag 21150:8@16+)
10.10.42.9 > 10.10.42.3: (frag 21150:4@24)
truncated-tcp 8 (frag 57499:8@0+)
10.10.42.9 > 10.10.42.3: (frag 57499:8@8+)
10.10.42.9 > 10.10.42.3: (frag 57499:8@8+)
10.10.42.9 > 10.10.42.3: (frag 57499:4@16)
truncated-tcp 8 (frag 57500:8@0+)
10.10.42.9 > 10.10.42.3: (frag 57500:8@8+)
10.10.42.9 > 10.10.42.3: (frag 57500:8@8+)
10.10.42.9 > 10.10.42.3: (frag 57500:4@16)
truncated-tcp 8 (frag 58289:8@0+)
10.10.42.9 > 10.10.42.3: (frag 58289:8@8+)
10.10.42.9 > 10.10.42.3: (frag 58289:8@8+)
10.10.42.9 > 10.10.42.3: (frag 58289:4@16)
The following is a comparison of how the tcpdump output from the F5 “fragrouter: frag-5: out of order 8-byte fragments, one duplicate” technique would appear against normal traffic. Note the Don’t Fragment (DF) flags on every packet of a normal connection, and note that the fragrouter stream has several fragmented packets.
Countermeasures
For those wishing to implement NIDS throughout their network infrastructure, fortunately there are some emerging technologies that help eliminate a great many of these lower-layer protocol vulnerabilities. Protocol normalization, as discussed by Mark Handley and Vern Paxson in May 2001 in “Network Intrusion Detection: Evasion, Traffic Normalization, and End-to-End Protocol Semantics” (www.aciri.org/vern/papers/norm-usenix-sec-01-html/index.html), is an attempt to scrub or rewrite network traffic as it enters a destination network. This scrubbing process should eliminate many of the difficulties in reconstructing a consistent view of network traffic. If an IDS and target host were both behind a network protocol scrubber, they would both receive an identical picture of the network traffic.
Using Application Protocol Level Evasion
IDS sensors have the ability to inspect the protocol internals of a communications stream to aid in the detection process. There are two basic strategies vendors employ: application protocol decoding, where the IDS will attempt to parse the network input to determine the legitimacy of the service request, and simple signature matching. Both of these approaches have their own unique challenges and benefits; we will see that most IDSs probably implement a hybrid of these solutions. Opportunities to evade detection are available at every layer of the protocol stack.
Security as an Afterthought
Application developers are typically motivated by features and dollars. We all know that the end user is the ultimate decision maker on the success or failure of software. In an effort to please end users, provide maximum compatibility, and eliminate erroneous conditions, developers omit strict compliance of protocol specifications in favor of error correction. It is uncommon for an application to immediately terminate requests upon the first deviation from specified protocols—quite to the contrary, every effort is made to recover from any error in an attempt to service each possible request (thereby maximizing compatibility and possibly increasing interoperability). As security researcher Rain Forest Puppy (known as RFP) stated at the CanSecWest Security Conference 2001, “You would be surprised with what passes for legitimate HTTP traffic …” These practices are the downfall of application security since they only serve to aid an attacker in allowing additional latitude in which to operate.
Evading a Match
Upgrades, patches, and variation of implementation may change the appearance (on the wire) of an application. Signatures—too specific, too general and just plain too stale—are a basic issue that continues to thwart IDS attack identification efforts.
If we look back towards our snort signatures, we can see quite clearly that one of them specifies the complete path name for the chgrp command. This signature is supposed to alert to the execution of some command through a Web server. Any attacker who is aware of the semantics for these rules could easily modify their attack to play any number of tricks in hopes of evading this match.
This rule itself is quite specific about the path and name for the chgrp command. We can plainly see that if the command resided in a different directory than /usr/bin, this signature would fail. Also, if the attacker were to simply ensure that their path environment variable were correctly set, they may just issue chgrp without the complete path to evade a signature match. Should the IDS be configured to alert when any of these variations are present? How many signatures would our IDS have if we were to account for these many variations?
Alternate Data Encodings
Largely implemented to support multiple languages, the standard text sent between a Web client and server may be encoded so that it’s interpreted as Unicode, which can represent any known symbol (the Unicode value for Yung is U+6C38). It also presents all new challenges to IDS vendors, as these values must be inspected and converted into ASCII for standard processing. This challenge is not that difficult to overcome; most systems implement a practice known as protocol normalization. Protocol normalization will take an input string and digest all known encodings, white space, and any protocol-specific delimiters in an attempt to produce the most basic form of the input.
Unfortunately, all the normalizations imaginable cannot overcome the challenge of monitoring closed source software packages. Without detailed information of the inner workings of a system, there can be no accounting for undocumented nonstandard features. Microsoft’s Internet Information Server (IIS) had one such special feature: %u#### encoding was allowed as an alternate to the normal Unicode encodings (%####). The famed Code Red worm had used this previously unknown technique to bypass many IDS signatures tuned to match for the specific .ida buffer overflow vulnerability. Lack of information is the worst enemy of a network defender.
Consider the following imaginary attack:
The attack here seems to exploit some Common Gateway Interface (CGI) application, and a simple signature is developed to alert to the known vulnerability. This signature would provide a very high-level assurance that there would be relatively few false positives, as the exploit-code is embedded right into the signature. However, we can see that if the attacker were able to send a modified attack string, through the use of some additional white space, they should be able to bypass a signature match. This exercise again illustrates the difficulty of signature development. If the signature left out a portion of the exploit code, there may be a great number of false positives, whereas if they embed some of the exploit code, the chance for evasion is greatly increased.
This is an incredibly simplistic example and is not that difficult to overcome. Adequate normalizations should be able to eliminate whitespace and allow for a signature match.
Web Attack Techniques
Several Web attack issues have been analyzed by Rain Forest Puppuy; see, for instance, “A look at whisker’s Anti-IDS Tactics,” from December, 1999 (www.wiretrip.net/rfp/pages/whitepapers/whiskerids.html). He has implemented a number of them into his whisker vulnerability scanner. We’ll take a look at some of them in the following sections.
Method Matching
The method of a HTTP request informs the server what type of connection to anticipate (GET, HEAD, POST, and so on). RFP found that many IDS signatures had completely failed to recognize any other methods. This is a somewhat depressing fact as many IDS vendors claim not to be totally dependent on signature matching to generate an alert.
Directory and File Referencing
A slash, the character that specifies a separation between directory and file names (/), can be represented in a couple of different ways. The simplest form is double or multiple slashes (/some//file.html = /some////file.html). These tricks will fool the simplest signature matches, providing there are no normalizations to counteract.
Another form of the same trick (this works only on IIS Web servers), is to use the DOS slash character (). If an IDS were not aware of this convention, it would not be able to generate a match.
These tricks work because they can reference a file by a different pathname. Amazingly enough, resolving a pathname is substantially harder then you would think (this is what has lead to a number of remote compromises in IIS, remember Unicode). Dot, the path to the current directory, and double dot, the path to the previous directory, can be used to obfuscate a file reference. An attacker may only need to use his or her imagination in constructing unique paths; all of these are equivalent requests:
GET /
/./some////file.cgi HTTP/1.0
GET /./some/////some/./file.cgi HTTP/1.0
A form of the aforementioned evasions is what RFP calls parameter hiding. This evasion is based on the assumption that some IDSs may only evaluate a request until it encounters a question mark (?), a hex-encoded value of %3f. This character is typically what will denote that any further parameters are arguments to a Web application. If the IDS simply wanted to alert to the request of a file, it may not fully evaluate the expression. The following two requests are equivalent:
Countermeasures
As discussed previously, a signature-based IDS may be able to normalize the communications stream. That is, as it inputs data destined for a HTTP server, it should apply some logic to reduce the input into its lowest common denominator (a single /, or resolving directory references). Partial signature matches may also help. If a sensor does not enforce a strong 100 percent match, they should be able to account for some variation of many exploit types.
Using Code Morphing Evasion
Polymorphism is the ability to exist in multiple forms, and morphing is the process used to achieve polymorphism. The objective of polymorphic code is to retain the same functional properties while existing in a structurally unique form. A NIDS has only the opportunity to inspect information as it exists on the wire; this would then only allow the structure of the exploit to be inspected. This feature had allowed viruses to remain undetected for quite some time. The only difference is that a virus scanner has the opportunity to inspect disk files instead of network data. The way that most virus scanning engines have tackled this problem is through the use of heuristic scanning techniques; this is similar to what a host-based IDS would do (identifying suspicious events, inappropriate file access, and so on).
Polymorphism is achieved through taking the original attack payload and encoding it with some form of a reversible algorithm. All of the NOP-sled instructions are substituted with suitable replacements. This encoded payload is then sent over the network with a small decoding function prefixed (this decoder is also dynamically generated to avoid a signature match). When the exploit runs on the target, the decoder will unwrap the original payload and execute it. This way, the original functionality is maintained.
Polymorphic shellcode is discussed thoroughly in this author’s paper that was released in early 2001 (www.ktwo.ca/c/ADMmutate-README). An engine is included for use in any current or future vulnerabilities. The basis for polymorphic code generation is that there is always more then one way to calculate a value. If, to exploit a vulnerability, we had to calculate the value of 4, we could do any of 2+2, 3+1, 6–2, and so on. There are literally endless methods to calculate a given value—this is the job of an exploit, the possessing of some machine instructions. To a NIDS examining network traffic, there is no way to identify 2+2 as being equivalent to 3+1. The NIDS is only given the low-level machine instructions to evaluate against a known pattern; it does not interpret the instructions as the target host will.
This technique has the ability to mask any exploit from detection, from any specific rule to the general. The only opportunity for a signature-based NIDS to formulate a match is if a signature for the small decoder is able to be determined. To date, I have not seen any signatures or techniques developed for this class of polymorphic shellcode. Table 16.1 shows a side-by-side view of two executions of a polymorphic shellcode engine.
As you can plainly see, there is very little correlation between the three executions. There are a huge number of permutations that can be used.
It is apparent that most IDSs are not always quite ready to run out of the box. They require frequent updating and maintenance to yield long-term success. The IDSs that do have hope of detecting unknown forms of attack are anomaly detection-based. These systems do not use signatures at all. They instead monitor all network communications as they occur and attempt to build a high-level image of typical traffic. A statistical anomaly would then trigger an alert. As the system matures and gains more entropy into its database, it would then theoretically become more accurate. There is some question whether or not a purely anomaly-based detection engine would be very effective, as exploit attempts seem to be quite normal in day-to-day network operation and may fall into the baseline of these systems. As in all things, a little of each is not a bad idea. A strong signature-based system supplemented by an anomaly-based detection engine should yield a high level of assurance that most intrusion events are monitored.
In the endless security game of cat and mouse, one can forecast the generation of polymorphic statistically normalized attack engines that should provide one more hurdle for NIDS developers to overcome.
Summary
Signature-based IDS sensors have many variables to account for when attempting to analyze and interpret network data. Many challenges continue to elude these systems. The lack of information that is available for inspection is difficult to overcome. However, the rate at which many IDS sensors have been maturing is quite promising; Gigabit speeds and flexible architectures supported by an evergrowing security community push forward to configure systems that are capable of detecting all but the most obtuse and infrequent attack scenarios.
At every layer of the network stack there are difficulties with maintaining a consistent view of network traffic, as well as the effect of every packet being transmitted. It is quite clear that an attacker has certain advantages, being able to hide in a sea of information while being the only one aware of their true intension.
Packet layer evasions have been well documented throughout the past several years. IDS vendors are quite aware of the many issues surrounding packet acquisition and analysis. Most networks are beginning to filter “suspicious” packets in any case—that is, any types with options and excessive fragmentations. Perhaps in the coming years, network layer normalizations will become commonplace and many of these evasion possibilities will evaporate.
The difficulty with analyzing the application layer protocols continues to cause ongoing headaches. Some proxy solutions have begun to take hold, but the bottleneck that these systems cause is often too great. They also suffer from similar issues as IDSs, unable to identify classes of attacks that they were not originally intended for.
It is quite acceptable to quash malformed TCP/IP packets in the case of an error; a legitimate end system would eventually retransmit. The same is not true for higher layers; a NIDS may have an extremely limited understanding of application protocols and the information they transmit. Polymorphic attacks present a significant challenge that cannot be easily solved with a purely signature-based system. These attacks may exist in virtually limitless combinations.
IDS evasion will continue to be a way of life on the Internet. There is an ever-renewing tide of tools and techniques that are developed and refined (eventually raising the everyday script kiddie into a more advanced skill set) to make the job of detection more difficult. One should continually monitor and investigate network activity to gain an understanding of what to expect during day-to-day operations.
Solutions Fast Track
Understanding How Signature-Based IDSs Work
The capabilities of a network intrusion detection system (NIDS) are defined by a signature database. This enforces the requirement for repeated updates to combat the frequency of new vulnerabilities.
Most NIDSs do not alert even to slight variations of the defined signatures. This affords an attacker the ability to vary their attack to evade a signature match.
Attackers will continue to vary their evasion techniques such that the processing required to monitor and detect is greatly increased. This would contribute to denial of service (DoS) attacks and evasion possibilities.
Using Packet Level Evasion
Many vendors implement Transmission Control Protocol/Internet Protocol (TCP/IP) with slight variations. A NIDS has a difficult time in constructing a view of network communications as they appear to other systems. This inconsistent view is what allows an attacker to evade detection.
Hosts may not adhere to Request for Comments (RFC) specifications and allow some packets where the NIDS may not.
NIDSs do not have enough information from the wire to reconstruct TCP/IP communications. With the options and states available in a TCP/IP stack, some ambiguities form as to how a host would interpret information; there is an insufficiency of information transmitted between systems when communicating.
Fragrouter and congestant are effective evasion tools. They implement a number of documented NIDS evasion techniques.
Using Application Protocol Level Evasion
Application protocols are verbose and rich in function. There are many subtle, antiquated and obscure application nuances that make effective application protocol decoding difficult. An attacker may compromise even the slightest oversight.
Applications tend to allow for slight variation; developers intentionally build in error-correcting cases that attempt to make sense of any request, no matter how malformed. With a lack of strict compliance to defined specifications, it is difficult for the NIDS to determine the behavior of a network application.
Multiple encoding options exist for data representation. Unicode, uuencoded, or hex-encoded options exist in many application protocols. These alternate representations complicate the development of detection engines.
Using Code Morphing Evasion
There is always more than one way to do something. When detection hinges on the identification of application code, there are many alternatives to code generation.
Most exploits will vary from host to host. Variations can be incorporated even when restrictions are placed on the length or type of codes possible.
Frequently Asked Questions
The following Frequently Asked Questions, answered by the authors of this book, are designed to both measure your understanding of the concepts presented in this chapter and to assist you with real-life implementation of these concepts. To have your questions about this chapter answered by the author, browse to www.syngress.com/solutions and click on the “Ask the Author” form.
Q: How many IDSs do I need to make them more effective?
A: All networks are different and require varying levels of monitoring. Your particular risk tolerance should help you find this out, though. A network that desires a high level of assurance that it is detecting many intrusion events should have at least one sensor per network segment (Layer 2). It is also desirable to have multiple vendor types implemented when an even higher level of security is needed (one vendor’s strengths would hopefully fill in gaps from another).
Q: Aren’t these techniques too advanced for most attackers?
A: Just like most other technologies, attack methodologies and techniques are eventually turned into boilerplate applications that anybody can wield. The layout of the virtual battlefield may change in an instant. The next big worm might wield these techniques, and force a sea-change in the IDS market.
Q: Where can I get information about new evasion attacks?
A: The “underground” scene is typically the catalyst for advancements in security technologies. Frequent online publications can be used to get a feel for where useful information may come from. There is no single source for where all new papers are distributed.
Check out the following sites, to start:
antisec (http://anti.security.is)
Phrack (www.phrack.org)
Packetstorm (http://packetstormsecurity.org)
Technotronic (www.technotronic.com)
Q: What do I do if I am inundated with alerts?
A: Secure systems rely on compartmentalization to attempt to contain intruders. If you see that you are being attacked at an abnormal pace, isolate and separate the troubled systems and try to identify if there are some hosts with well-known vulnerabilities or exposures. Correlate your logs and IDS events to give you a better picture of what may be going on. Do not rely on authorities and the network administrators of the attacking networks; they are usually far too overworked or uninterested to give a respectable amount of support.
Q: How do I know that my IDSs are working?
A: Ongoing auditing and testing should be done to ensure that networking systems are properly implemented. Independent reviewers should always be a part of secure systems to ensure that a fresh set of eyes is evaluating a network architecture and IDS implementation.