Overview of Intrusion Detection

Intrusion detection works in a similar manner to virus protection software applications. The virus scanner scans files within a given operating system and tries to match the file against a database of known viruses. If the file is matched against the database, the software application interacts and takes action. This action can be to remove the virus from the file or to delete the file from the file system of the operating system. Intrusion detection works in a similar way. Instead of checking files for viruses, an intrusion detection system monitors the flow of network traffic and compares this flow to the database of security signatures that have been configured on the intrusion detection device. When a match is made, the intrusion detection system usually can alarm, drop the packet, or reset the connection.

This provides an excellent security tool that can scan the traffic on the network in real time and take action against any suspect activity.

Two main types of intrusion detection systems are commonly available and in use in today's networks:

• Host-based intrusion detection systems

• Network-based intrusion detection systems

Host-Based Intrusion Detection Systems

Host-based intrusion detection systems exist on the actual hosts or servers that they are protecting. They use resources on the host, such as disk space, RAM, and CPU time, and run as any other application would. The IDS application installed on the host is referred to as an agent. The agent collects data by analyzing the operating system, applications, and system audit trails and compares this data to a predefined set of rules. These rules indicate whether a security breach or intrusion has been attempted. Because the agents actually run on the host, they can be fine-tuned to detect operating system intrusion attempts and offer greater flexibility in this area than network-based intrusion detection systems.

The host agents can usually be configured to report intrusion attempts locally by some client application or centrally to an enterprise monitoring system. Scalability always becomes an issue with host-based agents, as you must install an agent on each protected host. Figure 6-1 displays the deployment of host-based intrusion detection systems.

Network-Based Intrusion Detection Systems

Network-based intrusion detection systems are physical devices that are connected to various network segments within the protected network. Network-based intrusion detection systems usually comprise two components that work together to provide the IDS service. These two components are an IDS sensor (Cisco's is the Intrusion Detection Sensor) and an IDS management platform (Cisco's is the Intrusion Detection Director).

The IDS sensors are hardware devices that passively monitor and analyze the traffic flow within a network segment. The sensor monitors the traffic and compares the collected data to prebuilt IDS signatures, to build up a profile of activity on the network segment. One problem with the IDS sensors is their placement. They can only monitor traffic that their network connection sees. The network interface listens in promiscuous mode to process all network traffic, even that not destined for the sensor itself. The obvious problem is that a normal switch port creates a separate collision domain and a shared broadcast domain throughout the VLAN to which the switch port is connected. Therefore, the sensor only receives unicast traffic destined for the sensor itself and broadcast traffic on that VLAN. To get around this, you should connect the IDS sensor to what is called a Switched Port Analyzer (SPAN) port on the switch. SPAN ports can be configured on all of the Cisco Catalyst range of switches. A SPAN port can be configured to listen to all unicasts and broadcasts for specific VLANs on one port. This is ideal for the IDS sensor, as it can then passively monitor and analyze all unicast traffic on the network segment across multiple VLANs. Figure 6-2 shows a network-based intrusion detection system.

The second component of the network-based intrusion detection system is the IDS management platform. The IDS sensor sends notification messages to the IDS management platform, which can be configured to interpret these results and take necessary action on them.