IN THIS CHAPTER
Intrusion detection is one of the hottest areas in the information security landscape, with International Data Corporation (IDC) stating that the market has grown enormously the past couple of years (http://www.mindbranch.com/listing/product/R104-10352.html ). Although the promise of technology that automatically detects, alerts, and possibly stops hostile intruders is extremely attractive, the technology is also fairly young. This chapter provides the reader with a guide to intrusion detection system (IDS) products and introduces some of the ins and outs of implementing IDS solutions. The chapter also sheds some light on how to select the best IDS for your environment and your needs and gives you a realistic review of what these systems can and cannot do.
The term intrusion detection means many things to many people; however, for the sake of clarity, we’re going to define it as the act of detecting a hostile user or intruder who is attempting to gain unauthorized access. Assuming this definition, a number of popular methods are used to detect intruders—for example, inspecting system, Web, application, firewall, and router logs for hostile or unusual activity. Some system administrators implement binary integrity checkers such as AIDE or Tripwire in hopes of catching successful attackers when they deposit Trojan code on compromised servers. (Chapter 21, “Unix,” has further details on Tripwire.) Other administrators simply monitor event logs looking for failed user login attempts.
Although all these methods are helpful, they become difficult, if not impossible, to perform on a daily basis. Introduce a few hundred machines, and the task becomes downright overwhelming. Enter the intrusion detection system.
The roots of modern-day intrusion detection systems lie in the Intrusion Detection Expert System (IDES) and Distributed Intrusion Detection System (DIDS) models developed by the U.S. Department of Defense (DOD) back in the late 80s and early 90s. These were some of the first automated systems deployed. Today, most intrusion detection (ID) systems are designed with the same goal in mind: to help automate the process of looking for intruders. This can be as simple as the real-time parsing of firewall logs looking for port scans or as complex as applying inspection routines to raw network traffic looking for buffer overflow attempts.
Traditional IDS classification schemes put most systems into two distinct camps: misuse detection models and anomaly-based detection models. This chapter focuses on two implementations of the misuse detection (also called signature-based detection) model: network-based intrusion detection systems (NIDSs) and host-based intrusion detection systems (HIDSs). Readers should note that many other intrusion detection models do exist, but most of them are less popular. However, most modern-day IDS implementations can be grouped into one of these categories:
Network-based IDS—. In their current form, NIDS devices are raw packet-parsing engines—glorified sniffers on steroids. They capture network traffic and compare the traffic with a set of known attack patterns or signatures. NIDS devices compare these signatures with every single packet that they see, in hopes of catching intruders in the act. NIDS devices can be deployed passively without requiring major modifications to systems or networks.
Host-based IDS—. These systems vary from vendor to vendor, but they are usually system centric in their analysis. Most host-based IDSs have components that parse system logs and watch user logins and processes. Some of the more advanced systems even have built-in capabilities to catch Trojan code deployments. Host-based systems are agent based—that is, they require the installation of a program on the systems they protect. This allows them to be more thorough on some levels but also more of a headache to deploy and administer.
Anomaly-based IDS—. Anomaly-based systems are a bit more obscure and are oftentimes referred to as more of a “concept” than an actual model. The philosophy behind anomaly-based approaches is to understand the patterns of users and traffic on the network and find deviations in those patterns. For example, a user who normally logs in Monday through Friday but is now logging in at 3 a.m. on a Sunday might be flagged as a potential problem by an anomaly IDS. In theory, an anomaly-based IDS could detect that something was wrong without knowing the specific source of the problem.
The most common IDS types, both commercial and deployed, are HIDS and NIDS models. Anomaly-based IDSs are fairly new in the marketplace and are usually part of a Hybrid IDS that is part NIDS or HIDS and part anomaly-based.
NOTE
Many people unfamiliar with the intrusion detection system field confuse the technology with access control devices such as firewalls. Most intrusion detection systems, in their current form, do not serve as a method of access control. A number of them can be configured to interact with firewalls, but this is not their primary purpose. Beginners should think of an intrusion detection system as a type of burglar alarm and not as a lock or door.
Although IDS technology is certainly attractive, before sinking any time into IDS research, you should first ask whether an IDS makes sense for your organization. If, for example, an organization is lacking basic security fundamentals such as firewalls, system/OS lockdown procedures, or virus protection, an IDS deployment shouldn’t take priority over those efforts. An IDS should be installed only after other facets of the information security strategy have already been initiated (see Chapter 1, “Building a Roadmap for Securing Your Enterprise”) or to solve specific situations or shortcomings. For example, if a new e-commerce initiative is launched that you simply cannot secure adequately, an IDS might help you keep a sharper eye on it. In addition, some people use IDSs as validation tools for their firewall rulesets. But if your network is a chaotic collage of vulnerabilities, an IDS will simply help you become the master of the obvious. You’ll already have problems that an IDS certainly won’t fix. Remember, modern-day IDSs are still, for the most part, reactive devices (although they are starting to block attacks as well). They won’t fix your problems.
However, if deploying an IDS is in your future, the following sections should guide you to making informed product decisions.
Essentially, network-based IDSs are designed to inspect network traffic and look for known attack patterns, or signatures. They perform this task by examining each and every packet that traverses the monitored network segment. This is usually accomplished by putting the network interface card (NIC) in “promiscuous” mode and passing every frame to the running IDS process for analysis.
One of the most appealing aspects of the NIDS model is that NIDS devices are passive. In most cases, the rest of the systems don’t even know the NIDS devices are there. Even better, deploying NIDS devices doesn’t require the involvement of system administrators—a resource that becomes a stumbling point for large HIDS deployments.
Although the NIDS design is moderately effective, it brings with it a few interesting issues. For starters, the attack signatures are based on known attack types. This means that most attacks for which the devices have not been programmed will pass by unnoticed. This places the users of these systems one step behind in a game that most are already losing. For example, when the BIND NXT bug hit the Internet in 1999, no NIDS product on the market was capable of detecting the exploitation of the hole. Why? Because it was a new vulnerability and IDS programmers hadn’t built a check for it into their signature libraries yet. One by one, vendors began creating signatures for the attack, but most of these updates occurred weeks, sometimes months, after the attack was widespread.
TIP
For a description of the BIND NXT vulnerability, along with other BIND vulnerabilities, see http://www.isc.org/products/BIND/bind-security-19991108.html.
In short, if the IDS is not updated on a constant basis, it will be unable to help identify new attacks and trends—much like vulnerability assessment (VA) scanners. (See Chapter 11, “Vulnerability Assessment Tools (Scanners).”) Unfortunately, even when vendors and administrators alike work to keep NIDS devices up-to-date, they will still be one step behind.
Another issue that plagues the NIDS model is the widespread deployment of switched environments. Switches, by their very nature, reduce the amount of traffic that is sniffable. NIDS devices rely on their capability to view all network traffic, so if the NIDS devices can’t see the traffic, they can’t inspect it.
NIDS devices are also traffic sensitive. Although some of the higher-end NIDS systems are currently pushing the gigabit boundaries, some NIDS devices struggle in high-bandwidth environments.
Finally, IDS evasion via packet-mangling techniques and other clever tricks are big problems. Several papers and studies have been released detailing methods of fooling intrusion detection devices. These techniques revolve around resequencing attacks, fragmenting attacks, and performing other techniques that could be used to confuse NIDS devices. Vendors have responded to many of these issues, but evasion techniques will continue to be a problem for many NIDS solutions.
TIP
Security experts Timothy Newsham and Thomas Ptacek published one of the more famous IDS evasion papers in 1998. It continues to be referenced years later, because many vendors have failed to address the problems Newsham and Ptacek pointed out. You can find it at http://www.robertgraham.com/mirror/Ptacek-Newsham-Evasion-98.html.
The host-based model varies from the NIDS model on a number of fronts. First and foremost, the host model is a more intrusive one. Whereas NIDS devices are passive, host-based systems require agents to be installed on all monitored systems. This isn’t usually a problem on a few dozen machines, but placing agents on a few hundred— or worse, a few thousand—machines is no small chore. The potential overhead in deploying and managing a large number of agents has led many organizations to install the host-based models on critical machines and to use NIDS devices for the rest of the enterprise.
NOTE
Don’t assume that you are stuck with choosing between the host and network models. Most vendors agree that the ideal IDS deployment consists of an integrated approach, incorporating both host-based and network-based detection. In fact, this holistic model has been adopted by most vendors.
Some of the host-based models are more full-featured than others. For example, most products monitor system logs for basic events such as failed login attempts, the creation of new user accounts, or access violations. Some monitor kernel messages for certain types of activity that might be interpreted as hostile, and more advanced HIDS agents watch for the installation of Trojan code and back-door programs and even terminate rogue processes.
Although all these features sound cool, and can be quite useful, the host-based model is not without its problems. For example, HIDS agents are still limited by the same factors that plague the NIDS model: They primarily look for known problems. If a HIDS agent is programmed to spot Back Orifice (a well-known Trojan), it will most likely be successful in performing that task. When a new next-generation Trojan hits the scene, however, there is a good chance the HIDS agent will miss it until a product update/revision is released.
Another problem, albeit minor for most environments, is that of CPU load. HIDS agents are active processes and consume CPU cycles. On underutilized machines, this is a moot point. However, for machines that are squeezed for every single CPU cycle, a busy HIDS agent might be the straw that breaks the camel’s back.
Despite these shortcomings, one of the primary benefits of deploying host-based IDSs is the fact that they centralize and parse your system logs. This alone is a huge benefit because most organizations do not review their system logs until something goes wrong.
The other types of IDS are based on knowing the attack signatures that crackers use to attempt to break in. The underlying assumption of anomaly-based IDS is that your network has regular patterns of usage.
However, if you do not have regular patterns of usage, the anomaly-based IDS has a hard time functioning. An example of a network that would be a bad choice for an anomaly-based IDS is a university network where it is almost unpredictable what kinds of legitimate usage occurs on the network.
The way most anomaly-based IDSs work is that you have to put the IDS in training mode for a while for it to learn your network. Then, you switch it to detection mode. Every time it sees a new behavior on the network, you have to tell it whether that behavior is normal or not. Getting an anomaly-based IDS up and running is a very time-consuming process in the beginning and requires more personnel hours than other IDSs.
One of the better papers on the anomaly-based IDS appears in Phrack, of all places, and is available at http://www.phrack.com/phrack/56/p56-0x0b.
You should note two points above all others when reading this section. First, there is no “one size fits all” IDS solution on the market today, and I highly doubt there will be one anytime soon. The IDS product landscape is a diverse one. Products such as ISS RealSecure are easy to install and have a wide range of features, but they often fall over in high-bandwidth environments. Enterasys Dragon performs well and is liked by most Unix-savvy individuals, but its user interface and the learning curve associated with the product turn away most Windows-focused administrators. BlackICE’s raw power and simplicity might tempt some small organizations, but when you need to manage hundreds of thousands of events, Cisco’s IDS with the Cisco Secure Policy Manager (CSPM) is a much more manageable solution. In short, organizations need to understand what their parameters are and adopt a product that best serves those requirements.
Second, the product balances change almost yearly. For example, in one year, Cisco went from having one of the worst user interfaces (an HP OpenView hack) in the market to one of the best (CSPM). In three years’ time, the product known as ID-track from the company called Internet Tools was acquired by Axent, expanded upon, relabeled as NetProwler, and later acquired by Symantec when Axent and Symantec merged. NFR Security, Inc. (NFR) was way ahead of the IDS technology curve at one point and within three years was considered by most to be somewhat behind. The bottom line is this: Consider the comments in this text, the reviews that are published in magazines, and anything else you might find on the Internet, but be conscious of the age of the information. The issues will stay somewhat constant, but who and how they are addressed could change in as short a time as six months. Although products such as firewalls are fairly mature and are now mostly differentiated by features, speed, and price, the IDS market is anything but mature. The only thing that you can be sure of on the IDS front is that nothing will remain the same.
When choosing an intrusion detection system, understand that you are choosing two things: a) a product, and b) a partner (vendor) who will be updating that product. Although the vendor (or team, in the case of open-source solutions) behind the product is always a consideration, it becomes even more critical in the intrusion detection market. Because IDSs are so time-sensitive and dependent on product updates, a good system will become increasingly less useful if it is not attended to properly and regularly. Evaluating the vendor’s track record in regard to product updates is a worthy effort.
On the product side, several issues and features can be found in one IDS but not in another. However, many of the bells and whistles of these products are just that— cute features. Make sure that you evaluate the core components first, and then examine the bonus features. The following is a list of core components you should evaluate when making IDS selection decisions:
Depth of coverage—. One of the more important components of an intrusion detection system is its capability to detect a wide array of attacks. Although a great back-end engine, diverse customization options, and a slick management interface are all strong selling points, if the product is incapable of detecting more than a handful of attacks, it will do little good. Be sure that any NIDS solution you examine is bundled with a healthy set of attack signatures. On the HIDS front, be sure that the product does more than inspect a few log files for a handful of events, and make sure that the product supports all the platforms you need to monitor. If, for example, the HIDS agents support only Windows NT but you have both Solaris and Linux machines, you are going to come up short in regard to overall coverage.
Accuracy of coverage—. This is a hard factor to determine without thorough testing, but it should be noted that not all signatures have been created equal. False positives are a big problem with most NIDS solutions, and in large environments, these misfires can jeopardize the overall effectiveness of the intrusion detection effort. Products designed with the reduction of false positives in mind are always more desirable.
Robust architecture—. There are multiple components to an intrusion detection solution, and it is important that both the engines and the IDS framework itself have been designed with strength in mind. On the engine/agent side, products should be capable of withstanding both attacks and basic evasion techniques. Although evasion has traditionally been a problem that has plagued NIDS devices, and will most likely continue to trouble them for some time, insightful vendors have continued their attempts at addressing these issues. Less insightful vendors have chosen to ignore them, which not only reduces product effectiveness, but also reduces confidence among security professionals.
Scalability—. Multiple components affect IDSs on the “scaling” front, but the two biggest are in the areas of high-bandwidth monitoring and data management. The bandwidth issues apply to NIDS devices in that many products have problems monitoring high-bandwidth, high-session environments. On the management front, some products struggle with monitoring, storing, and presenting large volumes of alert data. For example, if you deploy a few dozen sensors (host- or network-based) on a high-traffic/high-alert network, they will be pumping a lot of data back to the centralized databases and consoles. Some back-end systems will crumble under such loads, or worse, the volume of data will make it incredibly hard for the security officers to sort through the alerts. However, it should be noted that these issues are not relevant in all environments. For example, if you are looking to place a few ID devices to watch over a few T1 connections, you aren’t likely to run into bandwidth and data storage issues.
Management framework—. Being able to detect attacks is crucial for an IDS, but equally important is the capability to clearly and efficiently present the data related to those attacks. If security officers are unable to access attack and alert data easily, the overall usefulness of the IDS is limited. When evaluating intrusion detection systems, be sure to use the management console in a live environment. Also, be sure you are comfortable with a system’s management framework, and make sure it allows you to easily access the information you want. In short, the management framework used to control and monitor the devices is almost as important as the HIDS and NIDS devices themselves.
Timely updates—. Much like in the vulnerability assessment (VA) and antivirus product fields, as new attacks continue to surface, the need for timely IDS product updates becomes critical. Operating an outdated IDS is analogous to operating an airport without radar. Although updates are a bigger issue in regard to NIDS products, the issue is still relevant to all IDS models.
Customizability—. Some intrusion detection products allow for a diverse range of customization, whereas others are fairly static and inflexible. For some organizations, customization features will not be a big issue, because they will be operating IDS solutions with out-of-the-box configurations. For others, customization is a must. However, when choosing an IDS vendor, you should evaluate your needs now, as well as in the future. Although you might not require the capability to write a custom signature today, you might need that functionality in the future.
Skill set requirements—. Intrusion detection devices should be treated like any other component of enterprise IT—properly trained staff should operate the solution. Unfortunately, the one thing both administrators and managers alike seem to cast aside are the issues surrounding IDS upkeep.
Several open source IDS solutions in the community are worth investigating. The most popular is Snort, created by Marty Roesch. Snort is often considered to be the Linux of the intrusion detection field. It touts a very active development community, a wide set of signatures, and a large base of deployed users. Snort uses an NIDS model and has a fairly extensive set of plug-ins and supporting applications. For example, the Carnegie Mellon CERT team has created a front-end Web interface for Snort called Analysis Console for Intrusion Databases (ACID). ACID provides administrators and security officers a more user-friendly view of Snort output, which is natively quite raw. Snort is actively being developed on Unix, although Windows ports of the application exist.
Two other popular open source IDS tools are SNARE and SHADOW—both are Unix based. SNARE (http://www.intersectalliance.com/projects/Snare/) is an HIDS, and SHADOW (http://www.nswc.navy.mil/ISSEC/CID/) has long been a favorite among government and military personnel.
TIP
You can find a good list of HIDS and NIDS tools at http://www-rnks.informatik.tu-cottbus.de/~sobirey/ids.html.
Although some components of open source IDS tools are a bit behind those of their commercial counterparts, portions of programs such as Snort are quite advanced. In the coming years, it will be interesting to see whether the successes of open source initiatives like the FreeBSD and Linux projects will be shared by the open source intrusion detection community.
The following is a list of some of the more common intrusion detection products on the market today. I’ve attempted to list all the major players, but please remember that the IDS industry is still quite young, with new vendors popping up periodically and others fading away or being bought and sold. Use this section as a guide, but be sure you do some online research when investigating IDS solutions.
Cisco is very active in the IDS market; it has a host sensor product as well as the NIDS products it has had for a few years. Its HIDS product runs on Solaris, Windows NT, and Windows 2000. Cisco is selling its NIDS as appliances—the idea is that you buy a fully contained box and drop it onto your network with a minimum of installation effort. Cisco is also one of the few vendors that sells an enterprise class as well as low levels for small shops. The NIDSs also support anomaly detection.
Vendor: Cisco Systems
Platform: Appliance, Solaris, Windows
Product: Cisco IDS
URL: http://www.cisco.com/warp/public/cc/pd/sqsw/sqidsz/index.shtml
Computer Associates bought the eTrust Intrusion Detection System, as well as the whole company, from Platinum. It also has several unusual features such as URL blocking to keep your employees from viewing Web pages they shouldn’t. The marketing for this product is definitely overboard; it has language like “complete protection” and “isolates, contains, and extinguishes all enterprise threats.” Needless to say, this is not really true. Discussions on the Net mention that it is very resource intensive.
Vendor: Computer Associates
Platform: Windows
Product: eTrust Intrusion Detection
URL: http://www.ca.com
Enterasys acquired the Dragon IDS with its acquisition of Network Security Wizards in 2000. Dragon is a Unix-based system that was built for easy monitoring of high-bandwidth environments. Dragon has traditionally been less polished than most IDS offerings, but it’s far more robust. Organizations with Unix expertise that require a highly customizable and extremely powerful NIDS solution will probably like Dragon a great deal. Those that have lower requirements or primarily Windows-based operations might be more comfortable with other offerings. Enterasys also offers a HIDS agent that ties into the Dragon framework.
Vendor: Enterasys Networks, Inc.
Platform: Appliance and Unix
Product: Dragon
Intrusion has an interesting history. It was originally known as ODS and provided network hardware. At the end of 1998, it got into the IDS business. In 2000, it changed its name to Intrusion.com, and then after the dotcom crash in late 2001, it became just Intrusion. Its product lines are the SecureNet NIDS appliances and SecureHost HIDS for Windows.
Vendor: Intrusion, Inc.
Platform: Appliance and Windows
Product: SecureNet and SecureHost
IntruVert is a relatively new player in the IDS game with the company founded in October 2000. IntruVert is focused on the high end of the market with products that can handle high-bandwidth enterprise class operations. It is a combination of an NIDS and an anomaly-based system.
Vendor: IntruVert
Platform: Appliance
Product: IntruShield
ISS was one of the first vendors to completely integrate its host-based and network-based offerings into a unified management framework. ISS RealSecure has one of the easiest management consoles to use, and its server and network sensors have matured greatly over the years. RealSecure runs on an assortment of platforms, including Windows and many versions of Unix, and has one of the broadest sets of attack signatures in the industry today. ISS seems to be making real progress: It recently certified its RealSecure product at gigabit speeds and now has a HIDS product as well.
Vendor: Internet Security Systems, Inc.
Platform: Windows NT, Windows 2000, Solaris, HP/UX, AIX
Product: RealSecure
URL: http://www.iss.net
ISS took over this product originally from Network ICE. It is a Windows-based solution that boasts a substantial signature base with an easy-to-use interface. BlackICE is also a favorite among home users, because watching all the attacks that come in through cable and DSL connections can be an enjoyable pastime. BlackICE’s biggest weakness has traditionally been in relation to its centralized console and management features. Running a single instance of BlackICE is easy; running a few dozen has traditionally been painful. It is a nice low-end product, though.
Vendor: ISS
Platform: Windows NT
Product: BlackICE
NFR has long been acknowledged as one of the first vendors to address many of the well-known NIDS evasion techniques. NFR’s Network Intrusion Detection System is a NIDS-based product that allows a high degree of customization through the use of ncode, a scripting language NFR developed. NFR’s product has traditionally been enjoyed by experienced IDS veterans. Although the product is capable of handling various types of packet-mangling attacks, it tends not to hold up in high-bandwidth situations. NFR has also added the NFS Host Intrusion Detection System to its product line.
Vendor: NFR Security, Inc.
Platform: Appliance
Product: NFR IDS
URL: http://www.nfr.com
nPatrol tries to do it all: It’s a NIDS and a HIDS with anomaly detection. It is a relatively new company, founded in February 2000.
Vendor: nSecure
Platform: Linux
Product: nPatrol
Symantec’s IDS strategy integrates its host-based system (Intruder Alert) with its network-based system (NetProwler). Although the two can be tied together using a common management platform, their similarities end there. NetProwler has traditionally possessed some interesting features not found in other IDSs, but it lacks some of the back-end support to handle many NIDS evasion techniques. NetProwler runs only on Windows NT, except for the Console which will also run on 2000, and a new version hasn’t been released in two years. In contrast, the HIDS Intruder Alert has one of the broadest offerings on the market in terms of supported operating systems.
Vendor: Axent/Symantec
Platform: Windows, various Unix versions
Product: NetProwler, Intruder Alert
URL: http://enterprisesecurity.symantec.com/products/products.cfm?ProductID=50
Intrusion detection technology can be a powerful ally. If organizations lack the capability to detect and respond to attacks, malicious users have a higher chance of successfully hurting these organizations. However, intrusion detection is far from a mature technology. Organizations looking to adopt IDSs should carefully gather their requirements, document their needs, and be prepared to support another operational system in moving forward. IDSs are not silver-bullet solutions, but their usefulness can be realized through intelligent deployment efforts.
The IDS Frequently Asked Question (FAQ) list. This is a great place for beginners: http://www.ticm.com/kb/faq/idsfaq.html. The ARACHNIDS database is an interesting open-source effort for community-driven IDS signature development: http://www.whitehats.com/ids/index.html.
International Symposium on Recent Advances in Intrusion Detection. A conference Web site of the yearly conference on intrusion detection. The papers are on this site, so it lets you keep up with the latest developments: http://www.raid-symposium.org/.
Talisker’s IDS page. Talisker keeps good track of all the latest IDS products and round-ups. If it’s an IDS product, he’ll have it listed: http://www.networkintrusion.co.uk/.
Two very informative mailing lists for in-depth IDS discussions are the Security Focus IDS list (Focus-IDS) and the IDS list run from the University of Australia. HTML archives can be found at http://archives.neohapsis.com/archives/sf/ids/ or http://archives.neohapsis.com/archives/ids/.