In the previous chapter, we used SNMP to query information from network devices. We did this using an SNMP manager to query the SNMP agent residing on the network device with specific tree-structured OID as the way to specify which value we intend to receive. Most of the time, the value we care about is a number, such as CPU load, memory usage, and interface traffic. It's something we can graph against time to give us a sense of how the value has changed over time.

We can typically classify the SNMP approach as a pull method, as if we are constantly asking the device for a particular answer. This particular method adds burden to the device because now it needs to spend a CPU cycle on the control plane to find answers from the subsystem and then accordingly answer the management station. Over time, if we have multiple SNMP pollers querying the same device every 30 seconds (you would be surprised how often this happens), the management overhead would become substantial. In a human-based example analogous to SNMP, imagine that you have multiple people interrupting you every 30 seconds to ask you a question. I know I would be annoyed even if it is a simple question (or worse if all of them are asking the same question).

Another way we can provide network monitoring is to reverse the relationship between the management station from a pull to a push. In other words, the information can be pushed from the device toward the management station in an already agreed upon format. This concept is what flow-based monitoring is based on. In a flow-based model, the network device streams the traffic information, called flow, to the management station. The format can be the Cisco proprietary NetFlow (version 5 or version 9), industry standard IPFIX, or the open source sFlow format. In this chapter, we will spend some time looking into NetFlow, IPFIX, and sFlow with Python.

Not all monitoring comes in the form of time-series data. Information such as network topology and syslog can be represented in a time-series format but most likely, this is not ideal. Network topology information checks whether the topology of the network has changed over time. We can use tools, such as Graphviz, with a Python wrapper to illustrate the topology. As already seen in Chapter 6, Network Security with Python, syslog contains security information. In this chapter, we will look at how to use the ELK stack (Elasticsearch, Logstash, Kibana) to collect network log information.

Specifically, we will cover the following topics:

• Graphviz, which is an open source graph visualization software that can help us quickly and efficiently graph our network
• Flow-based monitoring, such as NetFlow, IPFIX, and sFlow
• Using ntop to visualize the flow information
• Elasticsearch for analyzing our collected data

Let's start by looking at how to use Graphviz as a tool to monitor network topology changes.