Introduction

Botnets can be difficult to detect in a network, but recently, Portland State University’s Jim Binkley, a professor and network security engineer, modified a tool called ourmon to detect the presence of botnets using network traffic analysis. The basic idea is that ourmon detects network anomalies based on hosts that are attacking other hosts via denial-of-service (DoS) attacks or by network scanning. It can then correlate this information with IRC channels and tell you if an entire IRC channel (set of communicating hosts) is suspicious. Thus, it is possible to find an entire set of infected hosts at one time.

Ourmon is an open source tool. Originally, it was designed for network monitoring but after a period of time it was discovered that it was also an anomaly-based tool, meaning that once you knew what was normal, you could begin to get suspicious about what was abnormal (anomalous). Ourmon is a network-based tool and not a per-host tool like a garden-variety virus detector. It typically is used to tell you the state of all the hosts in an enterprise from one vantage point (the logical network center) and can be viewed as a statistical network trend indicator.

In this chapter and subsequent chapters we are going to take a look at various aspects of ourmon that pertain to low-level anomaly detection and higher-level detection of botnets. We will do this by looking at ourmon and how it works and also by looking at a few botnet-related case histories. Here is our chapter plan for the chapters on ourmon.

So the basic plan is to first look at four botnet-related case histories, and then discuss how ourmon works and how to install it. Then we proceed to the next chapter to look at the fundamental anomaly-based tools, which do not rely on IRC but simply look for “strange things” using statistics. Once we understand the anomaly-based tools we can take a look at the higher-level IRC-based statistics that can reveal botnets. Finally, we will take a look at some advanced data-mining tools and techniques that can help you differentiate borderline cases where, for example, it may not be clear that a given IRC host is due to malware, an IRC game, or possibly even a hacked host with an IRC channel used by a group of hackers for discussion or warez distribution.

Case Studies: Things That Go Bump in the Night

Before we take the plunge and give an overview of ourmon’s architecture, let’s first present four real-world case studies that we will reinforce as we go along. Here we will just briefly present some ourmon outputs in the form of reports or Web graphs and discuss them a little bit. Don’t worry if you don’t get all the details here. First let’s understand the big picture and details will emerge in later chapters. Each case study has a short name tag to go with it and there are four in all. One thing to point out is that all these cases are botnet related. We should also point out that all four cases have been collected from the Portland State University network. PSU currently has about 10,000 Ethernet switch ports with 26,000 students and faculty and a gigabit connection to the Internet. It’s a large network and can be said to be typical of larger enterprise networks.

Case Study #1: DDoS (Distributed Denial of Service)

Ourmon uses graphics based on Tobias Oetiker’s popular RRDtool system (http://oss.oetiker.ch/rrdtool). Figure 6.1 shows a typical RRDtool graph used in ourmon. In this case, the graph (or filter in ourmon lingo) is called the pkts filter, which shows how many packets per second (pps) the ourmon system is processing. It also shows whether the operating system and ourmon collection system are dropping packets. The system will drop packets when there is too much work to do and not enough time. In this case, we are not dropping packets. We see a daily stripchart, where the current time (now) is on the right-hand side and “moves” left based on ourmon’s cycle time of 30 seconds. In other words, the graph is updated twice a minute. Essentially, this is a normal graph and shows PSU’s normal daily traffic with an early afternoon peak of 60k pps.

Figure 6.1 Normal Traffic—Pkts Filter

TIP

It is important to understand what is normal in order to understand what is abnormal. You need to observe your ourmon graphs and data daily and over time build up some idea of what is normal. Then you will be able to spot anomalies.

In Figure 6.2 we see a very abnormal version of the pkts filter. This is a DDoS attack. Keep in mind that there are thousands of hosts contributing to this graph. However, it is also possible for one host to put a spike in the graph with a DoS attack.

Figure 6.2 External DDoS Attack

If you were the head network engineer and you looked at this graph, you might reach for the aspirin. There’s an anomaly now. Hopefully, you can spot it! Instead of the daily peak of 60,000 pps, apparently 870,000 pps have decided to show up for a brief time. The theoretical maximum for a gigabit Ethernet connection for 64-byte (minimum size) packets is on the order of 1.4 million pps. This is close enough (and bad). Ourmon and some human intelligence eventually got to the bottom of this attack. Apparently a student on campus was having a dispute with another person external to campus. The other person used a botnet to stage a multiple-system, large DoS attack on the PSU student’s IP host (and on port 22, the ssh port) for “revenge.” Many hosts (1000s) sent small TCP SYN packets to one PSU host. A botnet was used as the attack vehicle. This attack and similar attacks have damaged network services on campus at times in various ways. It is often the case that a DDoS attack will do damage to innocent parties by perhaps clogging up the Internet connection or causing network equipment to crash or suffer degraded performance. In fact, this attack caused ourmon to more or less stop during the attack because all the operating system could do was drop packets. The lesson here is that botnets can cause serious resource problems. We will return to this case study in Chapter 9 when we give some advanced techniques for interpreting ourmon data. One important lesson here: A remote DDoS attack via a botnet may take your network (or your network instrumentation) off the air.

Case Study #2: External Parallel Scan

In the next chapter (Chapter 7), we will talk about some fundamental tools that ourmon uses to detect anomalies of various kinds. These include scan detection tools. In Figure 6.3 we see a picture of a particular ourmon feature called the worm graph that graphs the number of internal (home subnet) or external network “worms” A “worm” in this case, doesn’t really mean hosts having viruses. It more or less means hosts exhibiting behavior you might expect from a worm. In ourmon, a host that scans is said to be wormy. We show scanners with a red color for outside to inside (them) and green for inside to outside (us). In this case we had a rather alarming scan with over 2,000 hosts from the outside to the inside. Again, this had to be a botnet. It was used to perform a parallel scan of PSU’s /16 address space. This graph sometimes shows parallel scans and sometimes shows DDoS attacks. In this case, data elsewhere showed that a hacker was looking for e-mail systems at port 25. This particular tool is related to the TCP port report, which we will discuss in the next chapter at length.

Figure 6.3 The Worm Graph—Parallel Scan

Notes from the Underground

Hackers Fingering Hackers

A graph like the one presented above once occurred because apparently hacker party A got mad at hacker party B and staged a botnet-based DoS attack on party B’s box and IRC server. However, party B’s box happened to be a very important but badly administered Windows server. The hackers didn’t know that the box had important administrative software on it. The administrators didn’t know that the box had hackers on it. The upshot of the massive DoS attack was that the administrators finally discovered that box B was hacked and fixed it.

Case Study #3: Bot Client

For our discussion of ourmon in this section, we are using two tables taken from an ourmon report called the IRC report (see Tables 6.1 and 6.2). This report is produced hourly and is a statistical analysis of various IRC channels seen in the packet stream. We have simplified the report and left out all kinds of details and numbers and reduced it to two tables for our discussion here. From a thousand miles up, the IRC report consists of a set of IRC channels (chat room names) and the IRC hosts that belong to those channels (IP addresses). Various statistics and sorts are presented for both channels and IRC hosts. Channels and hosts may be sorted in various ways including most messages in a channel.

Table 6.1 IRC Report: Evil Channel Sort

Table 6.2 IRC Report: Channel List for Channel Hobo

Our first table gives the evil channel sort. In this sort we rank channels high if they have more hosts in them with per-host higher-scanning weights. We will talk more about the scanning weight in the next chapter. For now, accept that we are just counting hosts (under the wormyhosts label). A scanner is a host that performed what appears to be an act of scanning. It is simply looking for other hosts—probably to attack them with an exploit. So for some reason channel lsass445 had eight scanners apparently out of 11 hosts. Given eight scanners out of 11 hosts in the channel including any IRC servers, it is pretty likely that this channel is a botnet. However, false positives do occur and a channel with just a host or two with a high scanner weight may easily turn out to be a false positive (not guilty). We call the scanning weight the TCP work weight and will talk more about it in the next chapter. We are also interested in the other three channels because they are borderline cases and far less easy to declare a botnet client network. Here it turned out that channels hobo and .i-exp were botnet channels with the same IP server address (we are not giving real IP addresses and will confine ourselves to giving addresses as either net 192.168/16 or 10/8. In our examples, addresses with 192.168 as a prefix may be assumed to be local. Addresses using net 10 may be assumed to be remote). It turns out that alien is innocent, and the other two channels are guilty. We will explain these details in Chapter 8 on botnets, and in that chapter and Chapter 9 give more details about how we investigated our data to determine if these channels were botnets.

Notes from the Underground …

From the enterprise perspective, you may encounter two types of botnet environments in your log files. The set of hosts participating in the bot traffic is called a mesh. You determine the type of mesh based on whether the botnet server is located inside or outside your enterprise:

 Client bot mesh This is the term for a set of botnet clients that exists within a campus or enterprise and communicates with an extenal botnet server. Botnet clients are sometimes called zombies.

 Server bot mesh This bot mesh includes an on-site botnet server. Botnet servers are sometimes called Command and Control (C&C) hosts.

Case Study #4: Bot Server

Case study #4 is about how we can detect an on-campus botnet server (C&C). Ourmon has the IRC report mentioned before and also a small set of RRDtool-based graphs, as seen in Figure 6.4. The graph shows the total network count of important IRC protocol message counts including JOIN, PINGS, PONGS, and PRIVMSGS. We suspect you can spot the anomaly. PING and PONG messages are used between servers and clients to maintain connectivity (JOINS too for that matter). Our normal count for PING and PONG messages is about 30 per sample period (a sample period is the 30-second fundamental ourmon sample time). All of a sudden PINGs and PONGS have gone way up. Wonder why? Simple. A botnet client was turned into a botnet server and all of a sudden had around 50,000 remote botnet clients. Our IRC report shows the amazing upsurge in connectivity as well. We will return to this botnet server case in a later chapter.

Figure 6.4 IRC Message Counts

Tools & Traps …

Botnet Servers and Clients

Botnet servers can have thousands of clients. Typical IRC channels used for chat by real human beings will not have that many clients. At our school we have never seen an IRC channel with more than 50 hosts in it used for real human chat. Thus, if you see an IRC channel with 36,000 hosts in it, you can be fairly sure you have a botnet server.

A botnet client is a piece of software. It may download a new version of itself. It can take commands from the C&C server. Thus, a botnet client can become a botnet server at any time. Or it can change its IRC channel, port, remote botnet server IP, and probably other attributes as well, including the set of attacks it uses. It is just software and it can always download a new version with more capabilities.

How Ourmon Works

In order to install and use ourmon, it is necessary to understand its architecture. In this section please refer to Figure 6.5, the Ourmon Architecture Overview, for our discussion. We will introduce some important configuration files and output files as we go along.

Figure 6.5 Ourmon Architecture Overview

First of all, we need to understand that as software, ourmon is a packet-sniffing system and it has to be hooked up to a network in such a way that it either gets all the packets via an Ethernet switch set up to do port mirroring (send packets from one port to the ourmon sniffing port) or via the older Ethernet hub technology that by default shares all packets on all Ethernet ports. We can call this setup network capture. It is also possible to run ourmon on a single host to just look at that host’s packets, which we might call host capture. This may make sense for an important server or for a host that for some reason you believe to be the target of hackers. Normally, however, ourmon is an enterprise-level tool and is used for watching all the packets in an enterprise (or all the packets in a server farm). We will assume an enterprise install in this book.

Ourmon has two big software pieces, which we call either

The probe is written in C and uses the libpcap library to read packets out of a kernel buffer. Libpcap (from www.tcpdump.org) is a library used in UNIX systems that hides the details of packet sniffing so that tools like ourmon and snort, which use it, are more portable. The ourmon probe is actually called ourmon (perhaps not a clever name) and is typically invoked at boot via a shellscript called ourmon.sh as follows:# ourmon.sh start

Or it can be stopped just as easily with:

The probe is configured via an ASCII configuration file called ourmon.conf, which is supplied but needs some customization. For example, it is important to set an enterprise home network address plus mask. This enables ourmon to determine if IP addresses belong to your enterprise or are external. The probe runs (we hope) forever and is typically started at the boot time for the system. The probe can run on Linux or FreeBSD. We prefer FreeBSD for heavy packet loads, but Linux will work. (We will talk in Chapter 9 a bit more about how to optimize the probe). The probe produces a set of heavily aggregated output files. These ASCII files are fed as inputs to the back-end graphics engine. The probe’s output files thus become the back-end graphics engine’s input files.

We should point out that as an optimization it is possible to install the probe on a separate box and then arrange for the output files to somehow be copied to the back-end graphics engine box. This enables you to devote more CPU to the probe host and also to isolate the Web server behind a firewall (out of your DMZ) if desired. The simplest installation is to put all parts of ourmon on the same host, though (which will be our assumption for this book).

One other point to make about the probe and the graphics-engine software is what we might call cycle-times. There are a number of cycle-times in the system. This concept is fundamental to network management and ourmon at base is a network management system that happens to do interesting anomaly detection as well. The probe runs in a 30-second cycle. In other words, every 30 seconds it generates a snapshot of packet inputs in its various output files (for example, the main output file for the probe is called mon.lite, but it’s just an ASCII file full of data). So basically the probe runs for 30 seconds, generates a bunch of statistics in various forms, and then writes those stats out and zeroes its counters, starting over. This gives us a view of the network that is shown in the back end that we can call the “current” view. This view never lags more than one minute behind what is going on now. So in summary the probe produces data at 30-second snapshot intervals. This is not a real-time view, but is typically described as “near-time” because it does not lag very far behind reality. Now we can talk about the graphics-engine software, which has some additional cycle times including hourly and daily summarizations for reports.

The graphics-engine software is driven out of a UNIX crontab script entry with three fundamental cycle times. Crontab is just a way for UNIX to schedule tasks. Once a minute a script called omupdate.sh is invoked that produces Web page/graphics and 30-second ASCII reports. This script actually does its work twice a minute so that ourmon can have its 30-second update of Web information. There are also scripts that run on the hour and one script that runs around midnight. Hourly scripts take 30-second logged information and produce hourly summarization reports. The midnight run takes the last hourly report of the day and creates a daily report. Ourmon keeps roughly a week of daily reports around. Not everything has a daily report, but there are a number of such reports that will be important for our botnet-related work. Figures 6.1, 6.2, and 6.3 (note the graph label “Daily”) are examples of 30-second outputs and are examples of the RRDtool sub-system as well. Table 6.1 for our botnet client case is an hourly summarization for the current day in the form of an ASCII report. Thirty-second summaries for IRC do exist as a report, but they typically aren’t very useful until summarized simply because IRC is a slow and sporadic communications mechanism.

The back-end graphics-engine software can be said to be organized around the cycle-time concept, which is related to an old network management notion called baselining. Baselining simply means we want the system to give us some notion of history about the data. For example, in Figure 6.4 we can see a week’s worth of IRC message counts and this lets us see at a glance that Friday was a bit exceptional. The Web server software portion produces two kinds of graphics, including RRDtool strip charts and top N talker histograms. In this book we neglect the histograms because they are more important for traditional flow-based network monitoring. However, the RRDtool strip charts have an interesting feature and that is that any RRDtool strip chart in ourmon comes as a set of four including daily, weekly, monthly, and yearly graphs. This enables us to baseline data over a year. ASCII report data, like the anomaly reports we look at in the next chapter, including the TCP and UDP reports and the botnet ASCII reports in Chapter 8, only have one week’s worth of data at the most. Put another way, if it’s not RRDtool data, we may have a 30-second report (now), an hourly summarization, and a daily summarization. We keep about one week’s worth of ASCII daily summarizations.

Roughly, the back-end graphics engine produces the following kinds of data placed on the Web:

The back-end graphics engine also stores two kinds of logging information. One directory called rrddata stores RRDtool data, which has a special format that enables the one year of baselining graphs to be created. The other directory, called logs, is where the back-end graphics engine scripts store all logging information for anything in ourmon that is not RRDtool related. For example, 30-second IRC reports from the front end are stored here and are then built into hourly summaries placed on the Web. We will return to the logs directory in Chapter 9 for some advanced data-mining techniques that can help us extract botnet-related IP addresses from data stored in some of the log directories. One important aspect of the ourmon log system is that in general it gets to a certain size after a week and doesn’t get any bigger. RRDtool logs have a fixed permanent size when first created, so they don’t grow over time either. The other kinds of data stored in the logs directory are rotated every day so that, for example, today becomes yesterday, yesterday becomes the day before yesterday, etc. The very last day is deleted. Thus the logs reach a rough size and don’t become an administrative problem.

Installation of Ourmon

In this section we will assume that you just downloaded the ourmon tar archive that will have a name something like ourmon27.tar.gz and you wish to install it. We must first discuss ourmon’s directory layout so that you know which files are which and where to look for data. We then discuss library dependencies, which are needed by both the probe and Web server software. Finally, we discuss running the configure.pl configure script that does most of the work and also touch on how to bring the system up.

With a few exceptions, ourmon is a self-contained system that can be installed anywhere on a UNIX system. The best approach is to unpack it (tar xzvf ourmon27.tar.gz) and then decide if you want the system where you unpacked it or if you want to move it somewhere else. Assuming you are happy with your install directory, it is a good idea to read the INSTALL file before you install it, and then run configure.pl, which will configure and install the system.

Important directories and files (for example, /home/mrourmon) are located within the base of the ourmon directory (see Table 6.3).

Table 6.3 Ourmon Directory Guide

Let us first talk about system dependencies. This means those libraries or facilities assumed by the underlying ourmon system. Because ourmon is an open-source system, we don’t install binaries. We give you the source code, and therefore a C compiler and Perl are understood as system components. In addition, there are four pieces of software that the open-source ourmon system assumes are available. Three of them need to be installed before you run configure.pl. The important system dependencies are:

Now we can run configure.pl, but first it is wise to be aware of what it will try to do and of the questions it will ask you. In general, when it asks a question you can go with the default (just press Enter), but sometimes you may want to change the answer to get things right. There are a couple of important questions that you want to get right. If configure.pl can’t find a compile-time dependency (like libpcap) for the probe, it will complain and stop. However, it doesn’t care if RRDtool is installed or not (you want to make sure that RRDtool does get installed). We will, however, give you a tip below for checking the RRDtool install. Roughly, configure.pl does the following things:

Before you run configure.pl, spend some time going over the configuration tips in the next section.

Summary

In this chapter, we have introduced you to the ourmon network management and anomaly-detection system. Ourmon is a free open-source tool downloadable from www.sourceforge.com. We also introduced you to four case histories that we will use to dig deeper into ourmon in the next three chapters. In addition, we discussed how ourmon works as a software system and looked into how to configure and install it.

In terms of botnets, we want to reiterate a few fundamental behavior patterns that we saw in our case histories. In our first case history we saw that a multiple host DoS attack might be launched from the outside aimed at a local server of some sort. We will return to this case history in Chapter 9 on Advanced Ourmon Techniques. This case history is disturbing, because large DDoS attacks are very hard to monitor and can cause a great deal of network distress. Our second case history is focused on large parallel network scans, and we will touch on how to get more details about such a scan in the Chapter 7 on anomalies. Of course, both our case histories show external attacks. Sometimes these attacks may be inside out and in that case they reveal serious signs of infected hosts in an enterprise. Ourmon’s anomaly system is both powerful and fundamental and a good understanding of it can help you fight botnets at least in terms of detecting attacking systems. Obviously, ourmon’s IRC mechanism may not always detect botnets or systems with a worm or virus because such systems may not use IRC or may lack a communication channel entirely. This is another good reason for understanding ourmon’s fundamental anomaly-detection subsystems. Our last two case histories are on botnet meshes, botnet client meshes, and internal (by definition) botnet server meshes. In Chapter 8 we will discuss ourmon’s IRC statistics and report features that can help you determine if you have attacking (and sometimes passive) botnet meshes of both kinds.

Solutions Fast Track