In the preceding section, we got a good understanding of what logs are like and the kind of data contained in them. I am sure that like any good investigator, we have a gut feeling that these can be pretty important. Let's work towards discovering exactly why this is so.
As we saw in the previous section, a log entry reflects an event that occurred in an organization's network. A group of log entries make a log file. Many such log files are directly related to the security, while others may have some entries specific to security-related matters. Security-related logs could be generated by anti-virus tools, firewalls, intrusion detection and prevention systems (IDPS), operating system, networking equipment and applications, and so on.
The key factors to understand is that logs are a human-independent record of system and user activity in a network. This makes them particularly unbiased and allows for court admissibility as evidence, provided that they are collected, handled, and preserved in an acceptable manner. Logs provide the telltale fingerprints in an incident. They can tell us what happened, the sequence of the events, which systems were affected/involved, what information was compromised/what was not, how did the users caused or responded to the incident, and the time frame all this occurred in.
Logs are broadly classified as security logs, system logs, and application logs.
Just about any organization worth its salt has a number of security measures in place. Each of these produces logs. These can include the following:
- Anti-virus/anti-malware software: This records the information related to various viruses, malware, rootkits, date and time of detection, systems it has been first detected in, disinfection/deletion attempt, quarantine action, and so on.
- Routers: Routers are usually the first line of defense in many networks. These are configured to allow or block specific network traffic based on the policies implemented by the network administrators. A study of the logs of blocked traffic can be useful from the forensics perspective.
- Firewalls: Just like routers, firewalls allow or block network activity based on the implemented policy; however, the methods used to examine the network traffic are much more sophisticated. This can include tracking of the state of network traffic and content inspection. The main difference between routers and firewalls lies in the more complex policies and more detailed activity logs vis-à-vis routers.
- Intrusion detection and prevention systems: As you learned in the previous chapter, intrusion detection systems (IDS) and intrusion prevention systems (IPS) identify, record, detect, and prevent suspicious behavior and attacks. As IPS systems are proactive, they may drop packets that are malicious in nature. Verbose logging of such packets is extremely useful from an investigation perspective. Logs of integrity-checking exercises performed by an IDS can also contain valuable information, especially when compared with the previous checks performed some time ago. This can help the forensic expert establish the time frame of the incident.
- Remote access software: Microsoft Remote Desk Protocol (RDP), LogMeIn, TeamViewer, and RADmin are a number of remote access software tools available that grant secured remote access through virtual private networking (VPN). A number of these VPN systems support granular access control (such as Secure Sockets Layer or SSL) and usually maintain detailed logs relating to the resource use that include date and time of access by the user, data transferred, as well as the details of all successful and failed login attempts. Information from these logs can help in identifying data theft and other remote activity, including unauthorized access.
- Web proxies: A number of organizations use web proxies to enable, monitor, filter, and restrict website access for their users. Web proxies also cache copies of the commonly requested webpages in order to make frequent accesses more efficient. Web proxies are designed to keep a record of all Uniform Resource Locators (URLs) accessed through them. As we can see, web proxies can help us in identifying where one of our users picked up an undesirable drive by download.
- Vulnerability management software: Thousands of vulnerabilities are discovered every year. This means that every organization has to constantly update and patch its environment. To do this on any scale requires specialized software for the job. This specialized vulnerability management software plays the dual role of patch management software and vulnerability assessment. Typically, vulnerability management system logs patch the installation history and vulnerability status of each host. This includes known/previously identified vulnerabilities in the software updates and may have information about the configurations of hosts. Vulnerability management software is usually run in the batch mode and generates a large number of log entries.
- Authentication servers: Whenever an entity (user or another computer) requires access to the network resources, its credentials need to be verified by an authentication server. Authentication servers include directory servers, single sign-on servers, an access point, or even a switch. These typically log every attempt at authentication: its origin, username, success or failure, and of course, the date and time.
Servers, systems, networking devices such as routers, and switches have their own operating systems. Each of these log a variety of security-related information. The most common are system events and audit records, as shown in the following:
- System events: From a forensic investigation perspective, system events such as starting a service, shutting down a system, or a failed event can be quite useful. All such events are usually logged with date and time, and could include status, error code, service name, and account associated with the event.
- Audit records: These store security information, including failed and successful authentication attempts, file creation, accesses, deletion, security policy changes, account creation and deletion, privilege escalation, and so on. The administrator can set the degree of detail captured in the logs.
Applications are used to create, store, access, and manipulate data. The operating system provides the foundation for these applications while the role of the security systems is to protect this data. While some applications produce their own logs, some use the logging systems of the OS to log the data of relevance.
Some of the data that can be found in application-related logs is as follows:
- Client/Server request and response: These can be critical in reconstructing the sequence of events. If successful user authentications are logged, it is possible to determine the user who made the request. E-mail servers record the sender, recipient, e-mail subject, and attachment details; web servers record requested the URLs and response provided by the server; and financial applications record the records that were accessed by each user. All this information put together can be a treasure trove of evidence in an investigation.
- Account-related information: Logs can contain information related to successful and failed authentication attempts, account creation and deletion, account privilege escalation, and actual use of privileges. These logs help in identifying events such as brute-forcing password attempts as well as identifying when and by whom was the application used.
- Usage-related information: Information such as the number of transactions per hour (or minute) and the specific size of each transaction (such as e-mail size, file upload size, and so on) can be found in such logs. We can use this to identify a sudden increase in e-mail traffic, which might be indicative of a virus or malware or the download of large files (such as movies) in violation of the company policy.
- Significant actions-related information: This can include application start, shutdown, failures, as well as major application configuration changes.