Chapter 1

Introduction

Abstract

This chapter provides an overview of traditional and passive network mapping. The chapter introduces the concept of Python Passive Network Mapping (P2NMAP) and examines the pros and cons of such an approach. The chapter also examines some of the dangers and pitfalls of active network scanning, especially in critical infrastructure based environments.

Keywords

Python
Passive
Network
Mapping
Nmap
Zenmap
ping
icmp
tcpdump
echo
reply

“Measure what is measurable, and make measurable what is not so.”

Galileo Galilei

Conventions Used in This Text

I use standard typographical conventions (bold, italics, etc.), to highlight text that stands out from the overall body of the paragraph. The font styles I will be using throughout the text are:
Italic
Used for file and directory names and to emphasize terms
Constant width
Used for code listings and script generated output
Constant Width and Bold
Used for user input
Enterprise Networks today are complex, difficult to investigate, require specialized tools and demand exceptional and expert skills in order to properly respond to incidents. When dealing with incidents that involve critical infrastructure or other regulated industry environments the specialization of the toolkits can indeed be daunting.
One of the first challenges that face incident response teams and forensic investigation units is “What does your network consist of and how is it configured?” This may seem like a simple question that is easily answered by the Information Technology group. However, when responding to incidents like Heartbleed, Operation Shady Rat, and breaches at major retailers, the technical information and details regarding the network map can be vital.
More specific questions may also include:
What internet protocol (IP) addresses and subnets do you operate?
What servers and end points are running?
Are the Servers local, hosted at an external site or in the cloud?
What Operating Systems are in use? What versions and are they up-to-date?
What Services (open ports) are available on each server and host?
What applications and databases are in use?
How is your network configured, protected and isolated?
What connections are allowed between servers, hosts and Internet users?
What connections have occurred recently?
Are the activities from or to specific end points anomalous?
Where are those connections (to and from)? If the connection include hosts outside the internal network where are these connections physically located in the world? Can they be pinpointed and verified?
image
Figure 1-1 Enterprise Networks.
If some or all of these questions can be answered the follow-up questions are of course:
How do you know? …and
Are you sure?
Typically these answers come from the Chief Information Officer (CIO) or the directed IT personnel responsible for the network along with the (Chief Information Security Officer) CISO and related cyber security staff members. Each of these groups utilize a variety of tools to assist in managing the cyber assets under their control. These tools can range from a simple set of spreadsheets to complex asset control inventory and management systems, or in the worst case, stored between the ears of the staff members themselves. Don’t get me wrong, many of these folks are very talented and have a pulse and deep understanding of the networks they manage. All of this information regardless of its source or form factor is important and valuable to incident response and forensic investigation teams. They of course have the arduous task of determining what is happening or has happened, who is doing it, how to mitigate and remediate the damage and better defend against future incidents. All of the data regardless of the means of collection however, is necessary to execute a comprehensive forensic investigation.
Python Passive Network Mapping: P2NMAP - An open source solution to uncovering nefarious network activity deals with the challenge “what does your network consist of and what are identifiable or unusual behaviors?” Traditionally, network mapping is an active process whereby IT and cyber teams utilize tools to identify network based assets.
Nmap, (Network Mapper - a security scanner originally written by Gordon Lyon - also known by his pseudonym Fyodor Vaskovich) used to discover hosts and services on a computer network, works by communicating raw IP packets to specified IP address ranges to determine:
what hosts exist within the range
what services are running on each of the discovered hosts
what operating system are those host likely to be running
…and a plethora of other characteristics that can be tested and measured through this active interrogation method.
By way of a quick introduction let’s take a look at the current instantiation of Nmap for Windows using the Zenmap Graphical User Interface (GUI).
Figure 1-2 depicts the main display of Nmap running under the Zenmap GUI version 6.47. Zenmap is a multi-platform graphical front-end that interfaces with the standard command line of Nmap and then displays the results in a more useable and interactive format.
image
Figure 1-2 Nmap Today.
As you can see in Figure 1-2A, I have selected a simple ping Scan with a target selection of 192.168.0.0/24. Zenmap displays the exact Nmap command that will be executed based on the selections that I have made. Dissecting the command reveals the specific instructions delivered to the Nmap engine.
image
image
Figure 1-2A Ping Scan Selection.
The results of this quick scan can be seen in Figure 1-3. As you scan through the list of computers and other devices on my local network you might find some interesting hits and responses.
1. IP address 192.168.0.7 was identified as a Roku box used for streaming content from the Internet.
2. IP address 192.168.0.7 was identified as our Bose Wave Radio.
3. IP address 192.168.0.13 was identified as our B-Link surveillance camera, just in case you had thoughts about stealing the Bose wave radio. :)
4. IP address 192.168.0.16 identified as an Apple device, ….this could be one of many
5. IP address 192.168.0.19 is a DirecTV receiver
6. IP address 192.168.0.185 is an Internet radio
Along with several other typical computers. (My wife was a computer scientist also… thus the ‘several’!) It is important to note, that the manufacturer identification of these devices is not based on any Nmap magic, but rather on the OUI (Organizationally Unique Identifier) portion of the MAC address.
This provides a pretty good scan of the active devices on my local network. Of course these are the devices that responded to scan. What about the printers and other mobile devices that were not identified? We will be discussing this issue throughout the book.
image
Figure 1-3 Summary Results of Ping Scan.
If you are a more visual person, Figure 1-4 provides a graphical view of the network IP addresses identified. This allows users to drill down into specific devices and discover additional information.
image
Figure 1-4 Network Map generated by Zenmap.

So What is a Ping Anyway?

Ping is the cyber equivalent of traditional SONAR (short for SOund Navigation And Ranging), or the “pings” that are used to locate objects under water. A cyber ping actually refers to the use of a special network protocol namely the Internet Control Message Protocol (ICMP). It is primarily used by network devices to send error messages indicating that specific services are unavailable or unreachable, or to communicate and query specific status.
For host discovery purposes, ICMP’s Echo Request message is used to make a request to a specific IP addresses and then wait for the associated Echo Reply Type Message. Traditional thinking is that if you cannot obtain a response from a host that you ping, other services offered are likely unavailable. In many cases when troubleshooting connection issues ping is used to verify connectivity to a specific IP address.
Due to increased concern and awareness of cyber security issues many network firewalls and gateways block ICMP Echo Requests to stop unauthorized mapping of hosts on the network. Unfortunately, this plays both ways as insiders that wish to add hosts to the corporate network will configure their systems to block ICMP Echo Requests as well and therefore will not be discoverable using this type of scan.
ICMP is part of the Internet Protocol Layer as shown in Figure 1-5 and ICMP messages are transmitted using IP datagrams as depicted in Figure 1-6.
image
Figure 1-5 Example Internet Protocol Stack Layers.
image
Figure 1-6 ICMP Message Contained Within and IP Datagram.
Many message types and codes exist as shown in Table 1-1 on the following page. For our use in host discovery, the highlighted Echo Request Type 8, Code 0 and Echo Reply Type 0, Code 0 represent our primary use. However, as you can see, ICMP has many other Types and Codes that are used by network devices. Note ICMP is an IP Type 1 message.

Table 1-1

ICMP Types and Codes

Type Code Description Query Error
0 0 Echo Reply (Ping Reply) image
3 Destination Unreachable image
0 Network Unreachable image
1 Host Unreachable image
2 Protocol Unreachable image
3 Port Unreachable image
4 Fragmentation Error image
5 Source Route Failure image
6 Destination Route Failure or Unknown image
7 Destination Host Unknown image
8 Obsolete image
9 Destination Network Blocked image
10 Destination Host Blocked image
11 Network Unreachable image
12 Host Unreachable image
13 Communication Filtered image
14 Host Precedence Violation image
15 Precedence Cutoff image
4 0 Source Quench image
5 Redirect image
0 Network Redirect image
1 Host Redirect image
2 Type of Service Redirect based on Network image
3 Type of Service Redirect based on Host image
8 0 Echo Request Ping image
9 0 Router Advertisement image
10 0 Router Solicitation image
11 Time Errors
0 Time to Live == 0 during transit image
1 Time to Live == 0 during reassembly image
12 Parameter Error
0 IP Header Error image
1 Option Field Missing image
13 0 Timestamp Request image
14 1 Timestamp Reply image
15 0 Obsolete
16 0 Obsolete
17 0 Address Mask Request image
18 0 Address Mask Reply image



To provide a quick demonstration, I configured a simple network made up of just 4 computers as shown in Figure 1-7.
image
Figure 1-7 Simple ICMP Test Network.
In this example, using the ping command, I sent ICMP Request Type Packets from 192.168.0.5 → 192.168.0.9. IP address 192.168.0.9 responded with the appropriate response message.
image
You might notice that the packet delays are timed and range from 40.6 ms to 279 ms. This may seem unusual to you. I chose this specific target IP address, (as shown in Figure 1-3, this is my Bose Wave Radio), to show the response to pings. As you can see responses from this device are a bit erratic in comparison to a typical desktop computer. Also, you may notice that each of the ICMP requests contain a different sequence number denoted as icmp req = 1, icmp req = 2, … icmp req = 6. This is because the ping command employs a monotonically increasing value starting at 1, since IP packets, by their definition, are unreliable (or, better defined, as best effort), and packets can be lost, respond out of sequence, or be delayed. Finally, you notice that the ping request includes a ttl value of 64, where ttl stands for Time-To-Live and is decremented by 1 each time the packet passes through a router. Therefore the ttl value set to 64 allows the packet to route to as many as 64 network hops before the IP packet would be discarded to avoid looping.
I also have setup 192.168.0.10 as a Linux Host running Tcpdump. Tcpdump is a network monitoring program that captures and records TCP/IP data. Tcpdump is primarily designed to capture packets, however, the program has many options that can also assist in filtering, and performing statistical calculations and provide users with information that can assist in determining the health of their network.
I utilized the following command line to execute the Tcpdump session:
$ sudo tcpdump –vv icmp
The sudo command pronounced (su “do”) allows some (or all) commands to be executed as root provided that the user has the appropriate privilege associated with their account. Tcpdump is the command that we wish to execute as root. The –vv option instructs tcpdump to provide verbose output and finally, the icmp designator instructs tcpdump to only capture icmp packets. The following is the abbreviated packet results captured by the tcpdump command.
TCP Dump Output
Request 1
image
Reply 1
image
Request 2
image
Reply 2
image
_______________________________________________________________ ……. Skipped for brevity
Request 6
image
Reply 6
image
Now that we have taken a quick tour of Nmap and have a fundamental understanding of a basic ping scan we will explore where this book will take us next.

What is Python Passive Network Mapping or P2NMAP?

Simply put, P2NMAP is a method to map networks using only the Python programming language without ever emitting a packet onto the network. In addition, we want our activities to be stealthy and not expose our investigation. This is not for hacking or nefarious purposes as you will see, but in many cases performing these activities without the perpetrators knowledge is important, especially when that perpetrator is an insider.
There are several advantages and some disadvantages of this method. Table 1-2 defines some of these advantages and disadvantages.

Table 1-2

Advantages and Dis-Advantages of P2NMAP

Advantages Disadvantages
Zero overhead or impact on the network itself. This can be very important especially within critical infrastructure environments, where activity scanning technologies can disrupt operations.
The time to compile a complete map of the network may take longer, although providing a more thorough view of the environment.
The ability to uncover hosts and services that are unknown or are missed by active scanning methods.
It is more difficult to identify details such as specific operating systems, hardware types and vulnerabilities.
Identify behaviors that are potentially dangerous, hostile, nefarious or outside of defined policies.
P2NMAP provides a full motion video in comparison with the snapshot approach that most active scanning methods provide.
P2NMAP provides an extensible framework where users can add new capabilities and extend behaviors using one of the most popular and easy to learn programming environments.

The Common Vulnerabilities and Exposure (CVE-2014-016) vulnerability (commonly referred to as “Heartbleed”) may be the longest running zero day vulnerability to date. It is important to note, that Heartbleed is NOT a vulnerability of the SSL protocol in general, but rather an example of an implementation bug. Once discovered, it has taken months to fully identify impacted systems, and even longer to remediate a solution. One of the reasons this is so is because to fully identify all the impacted systems, modern vulnerability scanners have to test every IP address and every possible port running on each of those systems. It is simply not enough to scan for common OpenSSL ports and then test for the vulnerability. Thousands of applications and services use OpenSSL and many do not use standard ports like 443.
When scanning for these applications and services the expectation of the scanners is that:
1. All the systems are powered on
2. The scanners have visibility and are not blocked by firewalls or guards
3. The scanning operations themselves won’t disrupt operations
4. The vulnerable services are in fact running
5. The vulnerable services are properly responding to the probes.
That is a lot of assumptions. In addition, if those systems are running inside a critical infrastructure environment good luck in convincing the operators to let you start wildly scanning every IP address and every port. Instead a more sanguine approach is to passively monitor these environments with zero danger in causing harm and a greater chance of identifying the full range of systems impacted by Heartbleed. You may say this might take weeks or longer to accomplish using a passive approach. However, ask the real operators of these environments, how long it took to actively scan these environments, how many scans were necessary, how many times systems and operations were disrupted and you will find, as the saying goes, that “discretion is the better part of valor”.

Why Does This Method Cast a Larger Net?

The simple answer is that you will find important and undeniable facts about how your network and environment is operating. By passively mapping the behavior of your network you will know, depending upon how long you monitor, every IP address that has touched the environment, what and where in the world they have touched, how often they have communicated, and at what time of day or night were they communicating. This can only be accomplished by patiently mapping these behaviors over time.
Much like cartography which is described as both the art and science of map making, network mapping requires the same discipline, patience and consistency. Unlike cartography, however, where maps are re-drawn every 50-100 years, the maps of our digital network can change dramatically in just days.
You can see the contrast between a modern network map and a cartographer’s map in Figure 1-8 and Figure 1-9, respectively.
image
Figure 1-8 Social Network Map.
image
Figure 1-9 Fra Mauro World Map circa 1480 AD.

How Can Active Network Mapping Actually Hurt You?

Active Network Mapping has several specific impacts:
1. Active network mapping behavior mimics hostile or hacking activity and can cause intrusion prevention systems to react to counter the actions.
2. Host based sensors can also identify these behaviors as hostile and react to the behavior and create outages.
3. Active scanning activities place significant load on the network, servers, routers and network devices.
4. Errors in setting up the scanners, (for example scanning improper IP addresses ranges), can inadvertently impact adjacent networks. If the resulting scan causes damage or outages to those networks, operators of the scanners can be liable.
One of my favorite examples of this comes from a release by Hewlett Packard in the midst of the discovery of the Heartbleed vulnerability:

“HP Integrated Lights-Out products (iLO, iLO 2, iLO 3, iLO 4) do not use the OpenSSL library and are NOT exposed to the CVE-2014-0160 vulnerability (now known as ”Heartbleed”) …. However, there is a bug in these libraries that will cause first-generation iLO and iLO 2 devices to enter a live lockup situation when a vulnerability scanner runs to check for the Heartbleed vulnerability.” http://h20566.www2.hp.com/portal/site/hpsc/template.PAGE/public/kb/docDisplay?docId=emr_na-c04249852-1&ac.admitted=1406398999314.876444892.199480143

The point is that by merely scanning these systems for the Heartbleed vulnerability you can literally shut the lights off.

Organization of the Book

In order to quickly address P2NMAP and get you started using, expanding and developing new innovations in passive network mapping, I have arranged the book to get to the point quickly. I would also like to provide detailed explanation of each step, script program and method, thus leaving nothing unexplored.
I want these processes to be easily usable by novice and expert users, students, academics, practitioners, programmers, incident response teams and those wanting to learn about both Python and network investigation as the same time. I always have found learning a new programming language or environment is much more fun if there is a problem to solve first.
In Chapter 2, I explain what you don’t know about your network - and more importantly, why you need to know it and why it is important. Also, I look at who is touching your network, and from where. Why should you be concerned about this?
Chapter 3 focuses on how to capture network packets with Python and some special tools. We also look at how you can efficiently store, index and manage what you capture. Most importantly, I discuss how you can do this silently.
Chapters 4 and 5 tackle the analysis of what we have captured, how to make sense of it and how to create an extensible toolkit. This toolkit can be freely used, shared, evolved and also includes opportunities for you to participate in the future expansion.
Chapter 6 takes a look at future opportunities and outlines next steps for P2NMAP.
Finally, each chapter includes a summary of topics covered, challenge problems and review questions making the book suitable for use in college and university academic environments.

Review

In this chapter we quickly examined Nmap and the basic method of scanning and mapping a simple network. We examined the ICMP protocol and demonstrated how ICMP Requests and Reply make up the ping operation that can identify IP addresses on your network. Through this process we showed how many devices not just computers are on your network and do respond to this door-rattling exercise. Next, I provided you with a quick overview definition of what P2NMAP is, and what some of the advantages and disadvantages to this approach are. I also took a look at why passive mapping can be safer and more thorough method for network mapping. Finally, we examined some ways that active mapping can actually be dangerous.

Summary Questions

1. What are the fundamental differences between active and passive network mapping?
2. What other specific harm could active network mapping cause and/or what regulatory policies could be impacted?
3. What advantages or disadvantages could be caused by passively mapping networks?
4. What benefits and/or limitations do you think choosing a language like Python might pose when applied to network mapping?
..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset
3.135.196.172