How Are Intrusions Detected?

An IDS has a special implementation of TCP/IP that enables it to gather the packets and then reassemble them for analysis. It is not enough to simply sniff the packets; an IDS must examine them. An IDS can use one of three methods to detect intrusion:

• Pattern matching or signature-based

• Statistical anomaly-based

• Stateful protocol analysis

A pattern matching or signature-based model uses a set of rules, or signature, to detect an attack in progress. A device used for intrusion detection is loaded with a set of signatures. Each signature contains information about the kind of activity to look for in traffic passing through the network to detect whether an attack is under way. When the traffic passing through matches the pattern contained in a signature, an alarm is generated, notifying the network administrator of the intrusion.

Statistical anomaly-based IDS rely on establishing thresholds for various types of activity on the network. This type of activity can lead to detecting an intrusion taking place. The drawback to this method is it is difficult to rely on solely; as the sophistication of the means of detecting an intrusion got better, so did the people attacking your organizations. Many attacks do not lend themselves to easily being detected based on threshold limits, which might require more manual intervention in the tuning process.

Finally, there is the stateful protocol analysis, or deep packet inspection, process. This is a process of comparing predetermined profiles of industry-accepted definitions of nonharmful protocol activity for each protocol (FTP, SSH, TFTP, and so on) state against observed behavior to located anomalies coming through your IDS. The thought process is that when a user begins a TFTP session, IDS should expect to see certain commands, and if it sees anything other than those basic commands (view, list, and so forth) it flags the activity as malicious.

Every IDS vendor (of which there are several) has buzzwords of every type to confuse the buyer on the explanation of how an IDS performs its job. This seems counter-productive because each vendor wants to sell, but alas, the world is a fickle place and security is full of snake oil! This section takes a high-level look at the methods any good IDS should use.

Signature or Pattern Detection

Signature or pattern-based detection compares known threat signatures to observed events to identify incidents. Signature/pattern matching is the most common method of detecting attacks, and it means that the IDS must recognize every attack technique to be effective. An IDS has large databases with thousands of signatures that enable the IDS to match attack signatures or patterns. This particular methodology is effective at detecting known threats, but largely ineffective at detecting unknown threats and many variants of known threats. Signature-based detection cannot track and understand the state of complex communications; therefore, it cannot detect most attacks that compromise multiple events.

For example, many IDSs are used to monitor abuse, such as a user visiting pornography or gambling websites while at work. Detecting that kind of misuse is based on a keyword; however, consider a different scenario in which someone uses ICMP to scan and map out your network.

This type of attack detection takes place at a more granular level than protocol analysis or anomaly detection. As a result, specific events are identified that, for example, indicate that a compromise has occurred. One of the most frequently matched patterns is when an attacker ensures that he has achieved root permissions on a host. The host replies that root access was achieved in a packet that will be sent to the attacker and that can be analyzed for the word root. This is a greatly simplified example, but it demonstrates what an IDS looks for (that is, matches).

Anomaly-Based Detection

Anomaly-based detection is used to compare definitions of what activity is considered normal against observed events to identify significant deviations. This method uses profiles developed by monitoring the characteristics of typical activity over a period of time. The IDS then compares the characteristics of current activity to thresholds related to the profile. Anomaly-based detection methods can be effective at detecting previously unknown threats. Common problems with anomaly-based detection are inadvertently including malicious activity within a profile, establishing profiles that are not sufficiently complex to reflect real-world computing activity, and generating many false positives that take a fair amount of time and effort to tune out.

Anomaly detection is similar to the training of spam filters because a period of learning by IDS allows it to determine normal baseline levels of activity. Of course, normal is different for every network. The thought behind this approach is to measure a baseline of statistics such as file activity, user logins, CPU utilization, disk activity, and so on. After the baseline is established, IDS is used to detect statistical anomalies.

For example, assume that you are monitoring activity and your IDS begins to note that, early every morning, many of the hosts on your network become active. You might not immediately know what is going on, but you are alerted that you should investigate.

Stateful Protocol Analysis

Stateful protocol analysis compares predetermined profiles of generally accepted definitions of benign protocol activity for each protocol state against observed events to identify deviations. Unlike anomaly-based detection, stateful protocol analysis relies on vendor-developed universal profiles that specify how particular protocols should and should not be used. It can understand and track the state of protocols that have a notion of state, which enables it to detect many attacks that other methods cannot.

Attacks use methods of altering the underlying protocol information to be successful. For example, the Ping of Death is successful because it alters the packet size and, through protocol verification, this would be detected. An IDS has a verification system that can flag invalid packets; this can include valid packets that are severely fragmented, which again proves that communication stream reassembly is important.

An important aspect of protocol verification is that of application verification, where the IDS detects inappropriate application protocol behavior. For example, the WinNuke attack uses NetBIOS (a valid protocol) but adds out-of-band information, which is valid but is used only to attack a host.

Problems with stateful protocol analysis include the following:

• It is often difficult or impossible to develop completely accurate models of protocols.

• It is resource intensive.

• It cannot detect attacks that do not violate the characteristics of generally acceptable protocol behavior.

Combining Methods

Attackers continually modify and improve their abilities, thereby making them increasingly difficult to detect. To combat this, IDS continues to evolve, becoming smarter and better at detection by combining the methods they use to detect intrusions.

For example, an IDS might have the capability to combine the methods of signature-based pattern matching, protocol analysis, and anomaly detection. This capability to use multimethod attack detection is another example of the ever-evolving way in which IDSs continue to grow.

Intrusion Prevention

An IPS picks up where an IDS leaves off by providing the capability to prevent an attack from being successful at the earliest possible moment by blocking, or rejecting, packets that match a particular signature or behavior. To make this effective, the IPS sits inline as opposed to using a network tap or port span. The most effective IPS, ideally, works with an IDS; many vendors have seen this need and combined the two technologies to make an IPS-capable IDS.

There are many types of IPS technologies, which are differentiated primarily by the types of events that they can recognize and the methodologies that they use to identify incidents. In addition to monitoring and analyzing events to identify undesirable activity, all types of IPS technologies typically perform the following functions:

• Stop the attack: The IPS can stop an attack by one of three means:

• Terminate the network connection or user session being used for the attack.

• Block access to the target from the offending user account source address (IP address).

• Block all access to the targeted host, service application, or other resource.

• Change the environment: The IPS could change the configuration of other security controls to disrupt an attack. Common examples are reconfiguring a network device to block access from the attacker or to the target, and altering a host-based firewall on a target to block incoming attacks.

• Modify the attack’s content: Other IPS technologies can remove or replace malicious portions of an attack to render it benign. A simple example would be an IPS removing an infected file attachment from an email and then permitting the cleaned email to reach its recipient.

IDS Products

Many IDS/IPS systems exist, and a lot of confusion surrounds them because there is little in the way of standards for how they operate. It is difficult to provide a direct comparison between products because terminology, meanings, features, and functionality have not matured to a level at which an effective comparison can occur. However, many products are based on the work done by the open source community efforts in the IDS arena. The foremost of these products is Snort (www.snort.org).

Snort!

Snort! Snort! Snort!

Don’t worry—the snorting that you are hearing is not coming from some sort of weird beast; it is coming from an open source IDS developed by Sourcefire. Following is a description of Snort, quoted from the website (www.snort.org):

...Snort is an open source network intrusion prevention and detection system (IDS/IPS) developed by Sourcefire. Combining the benefits of signature, protocol, and anomaly-based inspection, Snort is the most widely deployed IDS/IPS technology worldwide. With millions of downloads and approximately 300,000 registered users, Snort has become the de facto standard for IPS....

...Snort is capable of performing real-time traffic analysis and packet logging on IP networks. It can perform protocol analysis, content searching/matching, and can be used to detect a variety of attacks and probes, such as buffer overflows, stealth port scans, CGI attacks, SMB probes, OS fingerprinting attempts, and much more....

Snort uses a flexible rules language to describe traffic that it should collect or pass, and a detection engine that uses a modular plug-in architecture. Snort has a real-time alerting capability as well, incorporating alerting mechanisms for syslog, a user-specified file, a UNIX socket, or WinPopup messages to Windows clients using Samba’s smbclient.

Snort has three primary uses. You can use it as a straight packet sniffer such as tcpdump(1), a packet logger (useful for network traffic debugging, and so on), or as a full-blown network IDS.

The Snort application is an extremely well-written command-line application, albeit sometimes difficult to configure and monitor, unless you’ve cut your IT teeth on the command-line interface. To make it more click-friendly, several companies have made GUI interfaces to work hand-in-hand with Snort. The following figures show screen captures of a front-end third-party GUI application called IDScenter (version 1.1 RC4 is the most recent version); it is a GUI interface developed by a group of people at Engage Security (www.engagesecurity.com).

As shown in Figure 11-3, the IDScenter GUI front end enables users to have a graphical configuration and monitoring interface.

Figure 11-3 IDScenter Main Snort Configuration Screen

Figure 11-4 gives you an idea of the basic configuration options that can be set for a Snort operation. This screen in IDScenter is crucial for beginning the inevitable tweaking that must occur, and Figure 11-5 shows the method users have for selecting the intrusion or attack profiles that Snort will be required to look for and how to notify customers. In addition, Figure 11-5 shows the alerting options that can be set when situations that require administrative attention occur.

Figure 11-4 IDScenter Snort Rule Configuration Page

Figure 11-5 IDScenter Notification

You might already know about Snort if you are familiar with Linux or other *nix operating systems, but what you might not know is that the IDScenter software is Win32-based, as is the version of Snort that it manages.

Created by people who truly want Snort to work under Windows, a handful of Win32 Snort installation packages are available on the Internet. The majority of these packages work well, but a few of them need additional development cycles. Also keep in mind that after you configure and deploy a Snort machine, you cannot use it for anything else after you engage the monitoring functions.

Limitations of IDS

Still an evolving technology, IDS has some manageable limitations given its overriding benefits. An IDS should always be deployed in addition to prescreening routers and firewalls. Primary systems, such as firewalls, encryption, and authentication, are rock solid. Bugs or misconfiguration often lead to problems in these devices, but the underlying concepts are proven and accurate. Regardless, some of the limitations are as follows:

• HIDS versus NIDS debate: This should never be a debate; both are needed and should work together in a unified approach to increase your network’s security because they play different roles. Even though both tools are needed, each has its drawbacks and limitations. Network-based IDSs have two major limitations: switched networks and false alarms. In a switched network, network-based IDS must be able to view and analyze all network traffic of the network it is protecting to be effective. Because most networks use a switch of some form or fashion, a sniffer cannot see all the network traffic—only the traffic on the segment to which it is attached. This means you’d have to deploy a NIDS at the perimeter, similar to a honeypot (see Figure 11-6 in the next section); this protects you somewhat from external attacks but does nothing to protect against an internal attack. Furthermore, depending on how your NIDS is configured, you might end up with a high rate of false positives. Modern-day enterprise networks amplify this disadvantage because of the massive amounts of data that need to be analyzed. Host-based IDSs have a couple of disadvantages: complexity and failure due to compromise. First, the implementation of HIDS can get complex in large networking environments. With several thousand possible endpoints in a large network, collecting and auditing the cumbersome log files generated by each node can be overwhelming. Second, if the HIDS system is compromised, the host could cease to function, resulting in a stop on all logging activity. Furthermore, if the IDS system is compromised and the logging still continues to function, the trust of such log data is severely diminished.

Figure 11-6 Simplistic Honeypot Architecture

• Attack patterns and signatures: Similar to antivirus software that uses virus definition files (VDF) to remain current, IDS products use attack pattern signatures. These signatures range from the simple—checking the value of a header field—to the highly complex signatures that actually track the state of a connection or perform extensive protocol analysis. Signature capabilities vary greatly among IDS products and are not always updated with the latest attack signatures. Some network IDS products provide little ability to customize existing signatures or write your own, whereas other IDS products give you the ability to customize all their signatures and write almost any signature you can think of. Another important factor to consider is that some IDS products can check only certain header or payload values, whereas other products can give you the data from any portion of any packet.

• False positives and false negatives: A false positive is any normal or expected behavior identified as anomalous or malicious. The major problem that false positives create is that they can easily drown out legitimate IDS alerts. A false negative is any alert that should have happened but didn’t. False negatives produce two problems:

• There are missed attacks that will not be mitigated.

• False negatives give a false sense of security.

• Resource limitations: NIDS sit at centralized locations on the network. They must keep up with, analyze, and store information generated by potentially thousands of machines. NIDS must emulate the combined entity of all the machines sending traffic through its segment. This means if you have a large network segmented into 30 VLANs and you want to monitor traffic on all segments, you need a NID for each segment. You can purchase one NID and place it on the edge of your network, but then you capture traffic only coming in from potential external threats, and according to the NSA 93 percent of all attacks come from within your own network boundaries.

• Long-term state: A classic problem is slow scans, in which the attacker slowly scans the system. The IDS cannot store that much information over that long of a period, so it cannot match the data together.

• Sensor blindness: IDSs are built on regular computers that do not have any special capabilities; thus, it is possible to saturate the link to which they connect and blind them, thereby causing them to drop packets they should have been recording. For example, the open source port-scanning tool nmap includes a feature known as decoy scans, which causes nmap to send hundreds of scans using spoofed IP addresses. It therefore becomes an improbable task for the administrator to discover which of the IP addresses are real and which are decoy addresses. These two scenarios retain forensics data, however. If the attacker is suspected, the data is still there to find. Another attack is to fill up event storage.

• Storage limitations: When attackers try to blind the IDS sensor, they might have the dual purpose of filling up the sensor database or hard drive. This causes the sensor to delete events or stop recording events.

• Denial of service: An IDS is extremely complicated because it has an entire TCP/IP implementation running. As a result, IDS can be susceptible to attacks. Attackers can often download the same IDS their targets use free of charge, and then experiment to find packets that will disable the IDS. During the attack, the intruder then disables the IDS and continues the attack undetected.

• Fragmentation: The act of breaking up large packets into multiple smaller packets. The receiving TCP/IP stack then reassembles the packets and their data. Most IDSs do not have the capability to reassemble IP packets. Simple tools exist that can automatically fragment attacks to evade IDS.

• Pattern evasion/change: Many simple NIDSs rely on pattern matching. Attack scripts have well-known patterns, so simply compiling a database of the known attack scripts provides pretty good detection; however, it can easily be evaded by simply changing the attack script and thus rendering the IDS pattern match useless.

• IDS evaluation tools: Many tools are freely available to test the accuracy and usefulness of an IDS. The two most commonly used are snot and Stick. These tools create thousands of attacks to see whether the IDS can sense them. Attackers can use these tools to hide their attacks or to potentially blind the IDS.

These limitations do not mean that the uses of IDS are invalid or somehow lessened. Hacking is so pervasive and attack tools so readily available that it is astounding what an IDS can detect. Properly maintained and managed, IDS dramatically improves the security of any network. However, one of the key fundamental points of properly using an IDS has been saved for the end of this section.

A security policy is crucial to the successful use of an IDS. You might ask, “But why do I need another policy and process to follow? They are such a burden!” It cannot be emphasized enough that the assets your security measures are designed to protect have value, and not ensuring that everyone involved in protecting them follows the same standards would be a grave mistake. One cowboy can ruin it for everyone.