There are plenty of documented network security breaches that took place over an extended period of time. In these slow breaches, often times evidence indicates that there were signs and traces in both the server and network logs that indicates suspicious activities. The undetected activities were not detected not because there was a lack of information, but rather there are too much information. The critical information that we were looking for are usually buried deep in a mountain of information that are hard to sort out.
In this section, we will try to use Python to search through the syslog text in order to detect the activities that we were looking for. Of course, the exact terms that we will search for depends on the device we are using. For example, Cisco provides a list of messages to look for in syslog for any the access list violation logging, available at http://www.cisco.com/c/en/us/about/security-center/identify-incidents-via-syslog.html.
For our exercise, we will use a Nexus switch anonymized syslog containing about 65,000 lines of log messages:
$ wc -l sample_log_anonymized.log
65102 sample_log_anonymized.log
We have inserted some syslog messages from the Cisco documentation, http://www.cisco.com/c/en/us/support/docs/switches/nexus-7000-series-switches/118907-configure-nx7k-00.html, as the log message that we will be looking for:
2014 Jun 29 19:20:57 Nexus-7000 %VSHD-5-VSHD_SYSLOG_CONFIG_I: Configured from vty by
admin on console0
2014 Jun 29 19:21:18 Nexus-7000 %ACLLOG-5-ACLLOG_FLOW_INTERVAL: Src IP: 10.1 0.10.1,
Dst IP: 172.16.10.10, Src Port: 0, Dst Port: 0, Src Intf: Ethernet4/1, Pro tocol:
"ICMP"(1), Hit-count = 2589
2014 Jun 29 19:26:18 Nexus-7000 %ACLLOG-5-ACLLOG_FLOW_INTERVAL: Src IP: 10.1 0.10.1,
Dst IP: 172.16.10.10, Src Port: 0, Dst Port: 0, Src Intf: Ethernet4/1, Pro tocol:
"ICMP"(1), Hit-count = 4561
We will be using simple examples with regular expressions for our example. If you are already familiar with the regular expression in Python, feel free to skip the rest of the section.