• DHCP

• DNS

• FTP

• HTTP

• IRC

• POP3

• SMTP

• SOCKS

• SSH

• SSL

• SYSLOG

• Teredo

• GTPv1

File Carving

In an NSM environment, it is common to want to extract any files that were transferred from a session. If you have a PCAP file, there are tools that can do this (e.g., tcpxtract), but they usually operate on a byte level, without a lot of protocol knowledge. Once they recognize, say, the beginning of a PDF file, they can usually carve the next few thousand bytes and save them to disk, but this is only an approximation of the actual file that was transferred. This method may also require some manual carving after the fact to end up with the file you are looking for.

Bro, on the other hand, knows a lot about the protocols it decodes, and this knowledge makes all the difference. For our first example, we’ll look at a method to extract files out of packet captures with Bro. We’ll start by developing it as a tool we can call from the command line with a PCAP file, and then show how to integrate this into the Bro instance that runs with Security Onion so that it just runs all the time on our live network traffic.

First, let’s create a quick prototype. Bro has a built-in mechanism for performing on-the-fly analysis of files it sees going over the wire, called the File Analysis Framework. One of the possible “analysis” types it knows how to perform is to write the file to disk. You just have to ask Bro to do this for each file you’re interested in.

Fortunately, this is far easier to do than it probably sounds. Create a file called extract-files.bro that contains the following code:

# When Bro finds a file being transferred (via any protocol it knows about),

# write a basic message to stdout and then tell Bro to save the file to disk.

event file_new(f: fa_file)

{

 local fuid = f$id;

 local fsource = f$source;

 local ftype = f$mime_type;

 local fname = fmt(“extract-%s-%s”, fsource, fuid);

 print fmt(“*** Found %s in %s. Saved as %s. File ID is %s”, ftype, fsource, fname, fuid);

 Files::add_analyzer(f, Files::ANALYZER_EXTRACT, [$extract_filename = fname]);

}

This Bro script creates a new event handler for the file_new event that Bro creates whenever it sees a new file transfer begin, regardless of the protocol. The single parameter Bro passes to this is called f and is of type fa_file, a record type defined by the framework. Notice Bro’s use of the dollar sign character as the operator that refers to the fields in the record. The script references the id, source, and mime_type fields and uses those to construct and then print a short message about each file it finds. It then attaches the file extraction analyzer to that stream, which is the piece that actually tells Bro to save the file to disk. That’s it! Now that the ANALYZER_EXTRACT analyzer is attached to the file, Bro will take care of the rest.

All that’s left now is to actually run this against a PCAP file. Running a specific Bro script is very similar to the way we ran Bro before, but this time we just name the script file at the end of the command line. In fact, you can run multiple Bro scripts by just naming multiple scripts on the same command line, but we don’t really need to do that here. The command we will run is:

bro –C –r ../pcaps/bro-sample-traffic.pcap ../scripts/extract-files.bro

Figure 10.6 shows what our script looks like when it runs. Notice that each line contains the File ID, so you can find the corresponding log entry in files.log for more information if you need to.

After running the script, notice that you have all of the same log files that Bro generated in the previous examples. In fact, everything looks pretty much the same as before. But this time, there are files inside the extract_files subdirectory, as shown in Figure 10.7.

Selective file extraction

Looking at the output above, you can see there are a lot of GIF files in the PCAP I used for this example, and also some HTML text. These are all from web traffic, and not what most people think of when they think about “file downloads”. What if you’re not at all interested in those file types, and just want to extract only Windows executables? No problem! Notice from our earlier code sample that Bro actually knows the MIME type of each file. It figures that out by examining the file, rather than just trusting whatever the file transfer protocol said it was.

We can modify our script to check the MIME type for each file before we attach the extraction analyzer.

#!/usr/bin/env bro

# When Bro finds an executable file being transferred (via any protocol it

# knows about), write a basic message to stdout and then tell Bro to save

# the file to disk.

event file_new(f: fa_file)

{

    # Check that we have a MIME type value in this record

    if (f?$mime_type) {

      # See if the type is one we care about

       if(f$mime_type == “application/x-dosexec” ||

        f$mime_type == “application/x-executable”) {

           local ftype = f$mime_type;

           local fuid = f$id;

           local fsource = f$source;

           local fname = fmt(“extract-%s-%s”, fsource, fuid);

           print fmt(“*** Found %s in %s. Saved as %s. File ID is %s”, ftype, fsource, fname, fuid);

           Files::add_analyzer(f,Files::ANALYZER_EXTRACT, [$extract_filename = fname]);

      }

  }

}

This version is very similar to the first, but it adds an if statement to check whether the mime_type value exists in the file record (the f?$mime_type piece), and if so, checks the MIME type value before adding the extraction analyzer. Figure 10.8 shows the new output, skipping all the images and HTML content, and only processing executables.

In order to demonstrate another way in which Bro code can be run, let’s see a cool trick. So far, we’ve been explicitly running the Bro command and then naming the script we want to run. Notice the first line of the revised script starts with the typical Unix “shebang” (#!). As you might guess, this means that a standalone tool written in Bro can run from the command line just like any other scripting language. To do this, you must first change the permissions of the script to make it executable:

chmod 755 extract-exe-files.bro

Now, you can call the script like any other command, as shown in Figure 10.9. It still behaves the same way as before; creating log files and writing the extracted files to disk. It may be a more convenient way to run your code, though, especially if you’re providing this tool to other team members who may not be familiar with Bro itself.

Extracting Files in Live Network Traffic

Now that we’ve enhanced our script as a standalone tool, let’s take a look at what we can do to run it continually as part of Security Onion’s Bro instance. This would be useful if you wanted to extract every occurrence of certain file types in near-real time as they came across the network, rather than interacting with individual PCAP files retrospectively.

Typically, once you have a working Bro script, getting it to run all the time isn’t very difficult. When Security Onion runs Bro in sensor mode, Bro reads its configuration from /opt/bro/share/bro/site/local.bro by default. Of course, the term “configuration” is used pretty loosely here. Like everything else, local.bro is really just a script that Bro knows to load and run when it starts. The default version that ships with Security Onion really just loads other Bro scripts that perform some useful functions like detecting scans, logging applications in use on the network, adding GeoIP lookups to certain protocol logs, etc. Of course, local.bro really is just a “local” Bro configuration, and this is where you’ll add all your own customizations, too. The easiest way to get our file extraction code into Bro is to just paste it verbatim into the bottom of the local.bro file, as shown in Figure 10.10.

Whenever you make changes to local.bro, you need to run through a simple three-step process to make the running Bro instance aware of them. Each of the steps involves using the Bro control program broctl to perform a task. The commands for these steps are:

These commands and their output are shown in Figure 10.11.

After your new code has been running for a while, have a look in the Bro log directory /nsm/bro/logs/current. You can examine files.log to see what files were extracted. If any have been, you can look at the content of the “analyzers” field to see which files had the EXTRACT analyzer applied to them, as shown in Figure 10.12.

As you can see from the string’s output on the extracted binary, this file appears to be a copy of the Windows SSH client PuTTY.

This is probably a good time to mention that restarting Bro also forces it to archive any logs currently in /nsm/bro/logs/current and create a fresh new set. This also happens automatically each night (at 00:00 GMT for Security Onion’s configuration). If you’re working through these examples, don’t be surprised if some of your old logs disappear after you restart Bro, reboot the SO system, or even if you just leave things one day and come back the next. The logs are still there. Bro just moved them to a new date-coded subdirectory of /nsm/bro/logs.

Packaging Bro Code

Right now, Bro is configured to extract executable files from the live network stream, which is incredibly useful. These files can be further analyzed for signs of malicious logic, compiled for statistical analysis, or even have information about them fed back into Bro for further analysis. With that said, everything is working, but we could do a little more to package up our code into its own separate file.

If you think about it, continually adding new code directly to local.bro will probably become unmanageable after a while. Not only that, but if you ever want to share your scripts with other Bro users, you’ll have to comb through lots of unrelated modifications and make sure you get only the lines you need to share. This is likely to become a management pain after a while.

If you read the rest of local.bro (the parts we didn’t edit), you’ll see it really does nothing but load up other scripts, which are each stored in their own separate files. Let’s do this with our code!

As it turns out, this is extremely simple. First, cut all of our custom code from local.bro and paste it into a new file called extract-interesting-files.bro, and copy that file into the /opt/bro/share/bro/site directory. This is in Bro’s default load path, so we can just insert a statement like the following into local.bro to make our script run just like before:

@load extract-interesting-files

If you use broctl to check, install and restart the Bro instance, you’ll see everything is working just like before, but now our code is easier to find, manage, and share.

Adding Configuration Options

While our code is stored in a much cleaner way, the only problem now is that if we ever want to extract different types of files for some reason, we would have to edit the script, possibly introducing new bugs and certainly making it more difficult to track changes to functionality.

For the final revision of our script, we’re going to add a configuration parameter called interesting_types, which will be a set of MIME type strings we care about. Whenever Bro sees a file with one of those MIME types, it will extract it to disk. We will set up this parameter so that other scripts can modify its contents without actually editing the script file itself.

Here’s the updated version of our extract-interesting-files.bro file:

#

# A module to find files of interest (according to their MIME types) and

# log them to disk.

#

module ExtractFiles;

export {

  const interesting_types: set[string] = [

“application/x-dosexec”,

“application/x-executable”

                    ] &redef;

}

event file_new(f: fa_file)

{

  # Check that we have a MIME type value in this record

  if (f?$mime_type) {

      # See if the type is one we care about

      if(f$mime_type in interesting_types) {

          local ftype = f$mime_type;

          local fuid = f$id;

          local fsource = f$source;

          local fname = fmt(“extract-%s-%s”, fsource, fuid);

          print fmt(“*** Found %s in %s. Saved as %s. File ID is %s”, ftype, fsource, fname, fuid);

          Files::add_analyzer(f,Files::ANALYZER_EXTRACT, [$extract_filename = fname]);

       }

   }

}

Right away, notice that we’ve declared this as a Bro “module” called ExtractFiles. Bro’s modules are much like modules in other languages; they provide a new namespace for functions and variables, separate from the main namespace to avoid collisions. By default, these names are private to the module, so you see that we have to use the export directive around those variables and constants we want to make available to other namespaces.

We’re only exporting one name here, the constant interesting_types, which is defined as a set of strings. A Bro set is an unordered collection of items of whatever type you specify. You can add and remove items, as well as check to see whether an item is a member of the set. There’s only one real tricky part here. Even though this is a “constant”, it has the &redef tag at the end of the declaration. That means you can actually explicitly change the contents of the constant, though you will have to use the special redef statement to do it. This is Bro’s way of keeping you from accidentally modifying an important configuration parameter due to a bug in your code or any other type of mistake. We’ll see how to do this intentionally in a short while.

The last change we made to our code was to replace the conditional that checks whether this is a MIME type we care about. Before, we had the types hard coded, like this:

 if(f$mime_type == “application/x-dosexec” ||

  f$mime_type == “application/x-executable”) {

Now we’ve simplified it to just check to see if the MIME type is part of our “interesting” set:

 if(f$mime_type in interesting_types) {

Now, since we’ve already added the appropriate @load statement to local.bro, we should be nearly ready to go. Before we do that, though, let’s see how you can add new file types to extract. In Security Onion, Bro’s database of known MIME types can be found in the text files in the /opt/bro/share/bro/magic directory. Figure 10.13 shows how to modify local.bro to add new MIME types (GIF and HTML) to our “interesting types”. Notice that we’ve used the redef keyword to add two new values to the “constant” set (which, from the main name space, we refer to as ExtractFiles::interesting_types).

If you use broctl to check, install and restart the Bro instance, you should start to see a lot of files being extracted very quickly if you are monitoring HTTP traffic at all, as is shown in Figure 10.14.

Using Bro to Monitor the “Dark” Side

Extracting files is a very useful task for a variety of purposes, but the question you will probably be asking yourself at this point is, “But how can I get Bro to let me know when something interesting happens?” Let’s look at an example that asks Bro to look for a certain type of event and let us know when it finds one.

For this example, we’ll use Bro to implement a darknet detector. A darknet is any subnet (or individual IP, technically) that is unused in your organization and that no other hosts should have any legitimate reason to interact with. For example, if your organization is allocated a /16 network, you may have several /24 networks that are unused. If you grab a few of those and guarantee they won’t be allocated in the future, you just created a darknet.

Darknets are useful things to have, and are helpful for identifying internal scanning and reconnaissance operations, as might occur when you have a worm or an unauthorized individual running around inside your network. They are not entirely reliable, as it’s easy for legitimate users to typo an IP address or for legitimate services to be misconfigured to point to your darknets, but they provide valuable early warning and are worth the time to set up and monitor.

To begin, create the file /opt/bro/share/bro/site/darknets.bro and place the following code into it:

#

# This module allows you to specify a set of unused netblocks or addresses

# that are part of your internal network but are not used. When Bro sees

# traffic to/from these netblocks, it will generate a notice.

#

@load base/frameworks/notice

module Darknets;

export {

   # Create a notice type for logging

   redef enum Notice::Type + = { Darknet_Traffic };

   # Your darknets. This is empty by default, so add some network blocks

   # in local.bro. NOTE: You can add specific hosts here by specifying

   # them as /32 subnets.

   const darknets: set[subnet] = {} &redef;

}

# Check each new potential connection (successful or not, TCP/UDP/IP)

# against our darknet set

event new_connection(c:connection) {

   local darknet_conn = cat(c$id$orig_h, c$id$resp_h, c$id$resp_p);

   if(c$id$orig_h in darknets) {

    NOTICE([$note = Darknet_Traffic,

      $msg = “Traffic detected FROM darknet",

      $conn = c,

      $identifier = darknet_conn]);

   }

   if(c$id$resp_h in darknets) {

   NOTICE([$note = Darknet_Traffic,

      $msg = “Traffic detected TO darknet",

      $conn = c,

      $identifier = darknet_conn]);

   }

}

There are a few new things in here, so before we go on, let’s examine this code more closely.

First, notice that the code starts by loading the base/frameworks/notice module. This module is the implementation of Bro’s Notice Framework, a set of hooks that make it easy to create your own types of notices and to manage the notices of other modules. Before you can reference anything having to do with the Notice Framework, you have to make sure you’ve loaded it into your running Bro.

Next, you’ll see that we created a new set, called darknets. This set contains members of type subnet, which is a built-in Bro data type to store CIDR blocks, specified as literal values in the code, with the format x.x.x.x/y. By default, this set is empty, so just enabling this script won’t really do anything. We don’t know the allocated darknet subnet(s) in advance, of course, so this makes sense. We’ll configure this later, in local.bro.

Next, you’ll see that we provided an event handler for the new_connection event. Bro generates this event whenever it starts to track a new connection. The fact that this event was called doesn’t imply that the connection attempt was successful, as this event is called too early in the process to know that; only that a new connection was attempted.

The single parameter to the new_connection event is c, a record of type connection. Connection records are the kitchen sink of data that Bro tracks for connections. In addition to storing basic info like the source and destination addresses and ports, the connection state and Bro’s connection IDs, it’s common to find that other non-default or user-written scripts have stored additional data there, such as geotagging information, file hashes, etc.

Some of the most common pieces of data you will read from these records are the source and destination IPs and ports for the connection. Bro stores these as part of the id record, which is of type conn_id and a component of the connection type. Conn_id records have the following structure (Table 10.2):

Although we don’t need it for this example, it’s worthwhile to note that ports in Bro are native data types, and include both a numeric and a protocol piece. For example, you could use the following piece of code to assign the normal SMTP port to a variable:

smtp_port = 25/tcp;

For our example, though, we only care about the source and destination IP and the destination port, which we access through their nested data structures as c$id$orig_h, c$id$resp_h and c$id$resp_p respectively.

Our event handler starts by calling the cat() function, which simply takes the individual string representations of all of its arguments and returns them as a single string. Here, we’re building a string that consists of the source and destination IPs and the destination port. This is an easy way of creating an identifier for this connection. We’ll discuss why we need this shortly, but for now just know that we’re assigning this value to the variable darknet_conn.

Next, we examine the source IP (and later, in a nearly identical piece of code, the destination IP) to see if it’s in one of our darknets, like this:

 if(c$id$orig_h in darknets) {

 NOTICE([$note = Darknet_Traffic,

   $msg = “Traffic detected FROM darknet”,

   $conn = c,

   $identifier = darknet_conn]);

 }

This code says “if the source IP is in the set called darknets, generate a notice.” A notice is simply an entry in the notices.log file and is the most common way Bro draws extra attention to something. Bro also has the concept of alarms, which are like notices, except they go to the alarms.log file and are emailed out on a regular schedule. Notices can also be immediately emailed or paged for quick attention, but we won’t do that in this example.

In our code, we’re calling the NOTICE function, which takes exactly one argument: a record containing all the information about the notice. Although you can create a separate variable to hold the notice record, you will usually see code construct an implicit record using the [$field1 = value1,$field2 = value2,…,$fieldN = valueN] construct, which is the method we use here.

Each notice has a type, which is one of a set of enumerated values defined by the various modules loaded into Bro. The values themselves are not really important; they’re just used to differentiate one type of notice from another. You just need to know their names so you can pass them as the value of the $note field in the notice record. Each module that creates its own notices defines its own new types. Our code defines one type called Darknet_Traffic, like this:

# Create a notice type for logging

redef enum Notice::Type + = { Darknet_Traffic };

Each notice also includes a human-readable message in the $msg field. We’ve defined one notice type with two different possible messages, according to whether the traffic is detected going into or coming out of the darknet.

Next, we add the information about the current network connection to the notice record as the $conn field. Because Bro now knows which connection this notice is associated with, it can properly log the connection ID into the notice.log file, which builds the pivot linkages into the conn.log file that we saw earlier in the chapter.

Finally, our code adds the identifier we created earlier as $identifier. The identifier is important for generating notices with this script.

Extending the Darknet Script

With a little effort, the darknet module could be adapted to other uses as well. For example, if your security policy states that Active Directory servers should be on their own special subnet, and that the hosts on the network should not have access to the Internet, you could modify this code such that the list of “darknets” is replaced with a list of server subnets. In this case you would still expect to see a lot of traffic between your AD subnet and the rest of your internal network, so you can’t simply just alert on everything you see. With this in mind, you could either run this on a Bro instance that sees only traffic as it passes to and from the Internet (and thus should never see AD traffic anyway) or modify the logic of the new_connection handler a bit to make sure that the other side of the connection is not a valid local subnet (perhaps using a set of subnet types).

You could also use a similar approach to detect unauthorized connections between your DMZ and sensitive parts of your internal network (e.g. “why is the DMZ talking to the CEO’s computer?!”) or even replace the list of subnets with a list of allowed ports and audit connections using unusual or out-of-policy services. The possibilities for customizing and adapting this simple example to your own network are nearly endless, and really demonstrate some of the power and flexibility of Bro as a framework for detecting bad things on the network.

Overriding default notice processing

We’ve configured Bro to alert us whenever something tries to communicate with one of our darknets, but as written, our script isn’t very particular about exactly what is communicating with the darknets. I said before that there should be no legitimate reason for any other host on our network to try to talk to a darknet, but there are a few exceptions to this rule., such as, internal network mapping.

An organization sometimes wants to know what’s on its network, whether the purpose is to discover new devices that need to be managed or just to audit against unauthorized hosts being connected. Network engineers may do this on a regular basis, and it’s also a normal feature of many vulnerability management packages. So clearly there are some hosts that try to talk to our darknets, even if just to verify that they still really are dark. We don’t want to be alerted to all this legitimate activity, but as written, our script is likely to generate notices for an awful lot of this activity. Let’s see if we can fix that.

There are a couple of ways we could address this problem. Using only techniques we’ve already seen, we could modify the darknets module to define another set of the addresses of our authorized network scanners, then include some logic in the new_connection handler to check for set membership before generating a notice. That would work just fine, and would probably be a useful addition to the script. In fact, it’s probably the best way since we wrote the original darknets code ourselves. However, we’ve already seen all those techniques, so let’s try something new.

Let’s pretend that we originally got the darknet code from another Bro user. In that case, it might not make sense to directly modify the code, since we’d have to maintain a local patch, and apply and test it again with every new version of the module. That’s a pain, and fortunately, there’s another way. We can intercept Bro’s notices before they are written to the disk, inspect them, and decide whether we want to log the notice, discard it, or potentially take some other action.

We could put our new code into its own file, as we have been doing for the other examples However, by loading our code directly into local.bro, we keep the code that loads the darknets module and the code that modifies its logging together, which probably makes it clearer and easier to maintain for the purposes of this example. Here’s what the end of local.bro should look like, with both the previous darknets code and our new additions loaded in:

# Log notices for traffic to/from our unused subnets

@load darknets.bro

redef Darknets::darknets = [

            10.0.4.0/24,

          192.168.1.100/32

           ];

# These are our legitimate network scanners, which are allowed to talk to

# our darknets without logging notices.

const allowed_darknet_talkers: set[addr] = {

                   10.0.2.15

              };

# Process all notices at high priority, looking for Darknets::Darknet_Traffic

# types. When we find them and either the src or dst IP is an

# allowed_darknet_talker, remove all actions from the notice, which causes

# Bro not to log, alarm, email or page it.

hook Notice::policy(n: Notice::Info) &priority = 5 {

  if(n$note == Darknets::Darknet_Traffic &&

     (n$conn$id$orig_h in allowed_darknet_talkers ||

   n$conn$id$resp_h in allowed_darknet_talkers)) {

      # Remove all actions and assign it the empty set

       n$actions = set();

     }

}

The code starts by declaring a new constant, allowed_darknet_talkers, as a set of IP addresses (you could easily make this into a set of subnets if you have a particularly large number of network scanners or otherwise whitelisted IP addresses). I’ve added my scanning system’s IP (10.0.2.15) as the only member of the set.

Next, I’ve declared a new type of function, called a hook. Hooks are similar to event handlers, in that Bro calls these hooks as it’s processing traffic and doing work. The main difference is that events correspond to things Bro finds happening in network traffic, where hooks are called when Bro is performing its own internal processes.

In this case, when Bro generates a new notice, it calls the Notice::policy hook to allow you to perform local processing and modify how Bro will handle the notice (in other words, how it implements its notice policy for this notice). The single argument to this hook is n, a record of type Notice::Info that contains all the information Bro has about this notice.

One interesting thing about the hook declaration is that it uses the &priority keyword to assign a priority value of 5 to this hook function’s execution. Because you can have multiple event handlers for each event and multiple hook functions for each hook, Bro allows you to set the order in which they are called. Valid priorities are any integer between 0 (default) and 5. Hooks and event handlers are called in priority order, with larger numbers being called before smaller numbers. In this case, &priority = 5 helps ensure that Bro calls our hook function before it does any other notice policy processing, to give us a chance to bail out on processing this notice early.

The first thing our hook has to do is to decide if it should process this notice or not. There are many types of notices that Bro can generate, but we only want to concern ourselves with Darknets::Darknet_Traffic here, and only if one of the two IPs is in our allowed_darknet_talkers set. In our original Darknets module code, we already saw how we could use the Conn_id record type to access the source and destination IP addresses of the connection in question. The Notice::Info type also stores a Conn_id record that refers to the original connection which generated the notice, which we can access as n$conn. Therefore, n$conn$id$orig_h would be the client’s IP, and n$conn$id$resp_h would be the server’s IP. All this processing goes into a simple if statement at the beginning of the hook to make sure we meet all the requirements. If not, we simply don’t do anything, and Bro continues to log the notice as it normally would.

If, however, our conditional evaluates to true, Bro processes the body of the hook, a single line that assigns an empty set to the n$actions variable:

# Remove all actions and assign it the empty set

n$actions = set();

This requires some explanation. Bro decides what to do with a notice by examining the list of “actions” which have been assigned to it. Actions are just enumerated type values, and there are four to choose from. Table 10.3 lists these and explains what they do.

The underlying purpose of the Notice::policy hook is to allow you to change the set of default actions to be applied to the notice before Bro starts to implement those actions. In our code, since we removed all of the actions by assigning the empty set, we’re effectively telling Bro “do nothing with this notice.”

Now it’s time to try our new code. Recall that after restarting Bro, our log directory starts fresh and clean, so there are no notices. From our scanning host, we then ping one of the darknets (unsuccessfully) and connect to the printer port of the dark host we defined (successfully). But if we look at the log files, there are still no more notices (Figure 10.18). This indicates that our hook is working successfully, and will help us to avoid generating notices from traffic sources from approved internal scanning devices.

Suppressing, E-Mailing, and Alarming - The Easy Way

In the last section, we saw how to hook into Bro’s notice processing to provide detailed, granular customizations that allowed you to make decisions about how to process each individual notice. This surgical precision is nice, but sometimes all you need is a big hammer.

Bro provides a number of convenient shortcuts for changing the notice processing policy. These shortcuts take the form of constants you can modify to change the policy for all events of a certain type.

For example, suppose you have only a single SSH server on your network, and it’s exposed to the Internet. It has a legitimate business use (secure remote file transfers for your business partners), but the fact that it’s directly accessible from the Internet and needs to use the default SSH port means it’s going to be the constant target of password guessing attacks. Bro’s support for the SSH protocol logs the SSH::Password_Guessing notice each time it sees this type of activity, but you could get hundreds of these notices each day. This can be annoying since there’s nothing you can do to prevent these attacks. You won’t want to turn off Bro’s SSH protocol support entirely, but there is another option.

The answer is to add the SSH::Password_Guessing notice type to the list of notices which should just never be logged. Bro still tracks them (other notice types still depend on knowing when password guessing attacks are going on) but you’ll never see them. You can do this by adding the following to local.bro:

# Don’t generate any notices for SSH password guessing attempts.

redef Notice::ignored_types + = { SSH::Password_Guessing };

After you restart Bro, the password guessing notices will no longer show up in the logs. This affects all such notices from any host, so if you want to see notices from some hosts and not others, this mechanism is not for you. In our example, though, we have only a single SSH server anyway, so this is perfect.

The opposite example would be to escalate the processing of certain types of notices to alarms (alert data), or to email alerts. Assume you have at least one Internet-facing web server and are especially concerned with SQL injection attacks. Bro can detect SQL injection if you load the detect-sqli script (which is likely on by default), and will generate HTTP::SQL_Injection_Attacker notices whenever it sees a host performing such an attack against one of your servers (it also generates an HTTP::SQL_Injection_Victim notice for the target server, but we can ignore that for now).

The problem is, by default, these just go into the notice.log file with everything else, but you’d like them to go into alerts.log so you’ll get an automatic email summary every hour. You can accomplish this with the following code in local.bro:

# Alarm on any SQLi attempts

redef Notice::alarmed_types + = { HTTP::SQL_Injection_Attacker };

Alternatively, you can escalate even further, and turn these into immediate email alerts, like this:

# Send email immediately for any SQLi attempts

redef Notice::emailed_types + = { HTTP::SQL_Injection_Attacker };

Of course, for either of these last two to work, you’ll have to make sure you have defined the Notice::mail_dest variable discussed earlier. Otherwise Bro won’t know where to send the messages.

Adding New Fields to Bro’s Logs

For our final example, let’s circle back to the beginning of the chapter and talk a little more about logs. We’ve already seen that Bro does an excellent job of logging different types of transactions, but sometimes even Bro’s logging falls short. Perhaps you need to track an additional piece of information for certain transactions, or you maybe you just want to provide extra context around some events. Adding new fields to existing Bro logs is quite easy, and is a common thing to do.

Suppose you work for an organization that is concerned about where its Internet traffic is coming from or going to. One simple thing you can do to track this is have Bro look up the country codes for all network connections it sees (both sources and destinations) and add them to the conn.log file as new fields.

To start, paste the following code into a file called conn-geoip.bro:

redef record Conn::Info + = {

   orig_cc: string &optional &log;

   resp_cc: string &optional &log;

};

event connection_state_remove (c: connection)

{

  local client_geo_data = lookup_location(c$id$orig_h);

  local server_geo_data = lookup_location(c$id$resp_h);

  if(client_geo_data?$country_code) {

    c$conn$orig_cc = client_geo_data$country_code;

  }

  if(server_geo_data?$country_code) {

    c$conn$resp_cc = server_geo_data$country_code;

  }

}

The script begins by adding two fields to the Conn::Info record type. This is the data type Bro uses to store information about each connection it sees. It typically records things like the connection timestamp, uid, endpoints, application-layer protocol, and so on. In our case, we’re adding two new fields to store country codes for the endpoints, orig_cc and resp_cc.

Notice that each of the new fields is tagged with a couple of options. These tags are not part of the record itself, but rather, they tell Bro how to treat these fields in various circumstances. The &log option tells Bro to make sure to write the value of this field to the log file when it creates the conn.log entry. If you leave this out, Bro will track the data but you’ll never see it in the log file. The &optional tag specifies that it’s OK for this field to have no value (for example, if one of the endpoints is an RFC 1918 address and thus has no specific geographic ties). In this case, Bro will just log a “-“ in that field, which is the default behavior for any field in any log if there is no value to record. Technically, you could replace &optional with &default=“None” (or some other string) if you wanted to log a different value in that case, but “-“ is the standard everywhere else, so we’ll stick with that for consistency.

Next, we set up a simple event handler for the connection_state_remove event, which Bro generates just before it’s ready to remove an active connection from its state table and write the log to disk. We’re using the lookup_location() function, which is built in to Bro. It takes an IP address as its argument, and returns a geo_location record, which Bro defines like so:

type geo_location: record {

  country_code: string;

  region: string;

  city: string;

  latitude: double;

  longitude: double;

}

Our script generates two lookup requests, one for each side of the connection, with the IPs stored in the connection record:

local client_geo_data = lookup_location(c$id$orig_h);

local server_geo_data = lookup_location(c$id$resp_h);

Next, it checks each return value to see if the resulting record includes country code information. If so, it assigns the country code to the appropriate field in the newly redefined connection info record. Otherwise it does nothing.

if(client_geo_data?$country_code) {

  c$conn$orig_cc = client_geo_data$country_code;

}

That’s it! Once we added the two new fields to the Conn::Info record and set the &log parameter, we guaranteed that those values would be logged to the conn.log file. All the connection_state_remove handler had to do was to look them up and insert them into the connection info record. Bro handled all the rest.

Now it’s time to run our script. For demonstration purposes, we’re going to go back to testing it from the command line with a PCAP file, though you now know enough to permanently add this to your running Bro instance, if you wish. Figure 10.19 shows the tail end of our connection log now, extracting IP address and geographic data for each endpoint.