There are plenty of documented network security breaches that took place over an extended period of time. In these slow breaches, quite often, we saw signs and traces in logs indicating that there were suspicious activities. These can be found in both server and network device logs. The activities were not detected, not because there was a lack of information, but rather because there was too much information. The critical information that we were looking for is usually buried deep in a mountain of information that is hard to sort out.

In this section, we will try to use Python to search through the Syslog text in order to detect the activities that we were looking for. Of course, the exact terms that we will search for depend on the device we are using. For example, Cisco provides a list of messages to look for in Syslog for any the access list violation logging. It is available at http://www.cisco.com/c/en/us/about/security-center/identify-incidents-via-syslog.html.

For our exercise, we will use a Nexus switch anonymized syslog file containing about 65,000 lines of log messages this file is included in the accommodated book GitHub repository for you:

$ wc -l sample_log_anonymized.log
65102 sample_log_anonymized.log

We have inserted some Syslog messages from the Cisco documentation (http://www.cisco.com/c/en/us/support/docs/switches/nexus-7000-series-switches/118907-configure-nx7k-00.html ) as the log message that we should be looking for:

2014 Jun 29 19:20:57 Nexus-7000 %VSHD-5-VSHD_SYSLOG_CONFIG_I: Configured from vty by admin on console0
2014 Jun 29 19:21:18 Nexus-7000 %ACLLOG-5-ACLLOG_FLOW_INTERVAL: Src IP: 10.1 0.10.1,
 Dst IP: 172.16.10.10, Src Port: 0, Dst Port: 0, Src Intf: Ethernet4/1, Pro tocol: "ICMP"(1), Hit-count = 2589
2014 Jun 29 19:26:18 Nexus-7000 %ACLLOG-5-ACLLOG_FLOW_INTERVAL: Src IP: 10.1 0.10.1, Dst IP: 172.16.10.10, Src Port: 0, Dst Port: 0, Src Intf: Ethernet4/1, Pro tocol: "ICMP"(1), Hit-count = 4561

We will be using simple examples with regular expressions. If you are already familiar with the regular expression in Python, feel free to skip the rest of the section.