We import a number of libraries to assist with argument parsing, writing CSVs, processing event logs, and the custom pytskutil module.
from __future__ import print_function
import argparse
import unicodecsv as csv
import os
import pytsk3
import pyewf
import pyevt
import pyevtx
import sys
from utility.pytskutil import TSKUtil
This recipe's command-line handler takes three positional arguments, EVIDENCE_FILE, TYPE, and LOG_NAME, which represents the path to the evidence file, the type of evidence file, and the name of the event log to process. Additionally, the user may specify the directory within the image to scan with the "d" switch and enable fuzzy searching with the "f" switch. If the user does not supply a directory to scan, the script defaults to the "/Windows/System32/winevt" directory. The fuzzy search, when comparing file names, will check whether the suppled LOG_NAME is a substring of the filename rather than equal to the filename. This capability allows a user to search for a very specific event log or any file with an .evt or .evtx extension, and anything in between. After performing input validation checks, we pass the five arguments to the main() function:
if __name__ == "__main__":
parser = argparse.ArgumentParser(
description=__description__,
epilog="Developed by {} on {}".format(
", ".join(__authors__), __date__)
)
parser.add_argument("EVIDENCE_FILE", help="Evidence file path")
parser.add_argument("TYPE", help="Type of Evidence",
choices=("raw", "ewf"))
parser.add_argument("LOG_NAME",
help="Event Log Name (SecEvent.Evt, SysEvent.Evt, "
"etc.)")
parser.add_argument("-d", help="Event log directory to scan",
default="/WINDOWS/SYSTEM32/WINEVT")
parser.add_argument("-f", help="Enable fuzzy search for either evt or"
" evtx extension", action="store_true")
args = parser.parse_args()
if os.path.exists(args.EVIDENCE_FILE) and
os.path.isfile(args.EVIDENCE_FILE):
main(args.EVIDENCE_FILE, args.TYPE, args.LOG_NAME, args.d, args.f)
else:
print("[-] Supplied input file {} does not exist or is not a "
"file".format(args.EVIDENCE_FILE))
sys.exit(1)
In the main() function, we create our TSKUtil object, which we will be interacting with to query the existence of the user-supplied path. If the path exists and is not None, we then check whether fuzzy searching has been enabled. Regardless, we call the same function, recurse_files(), and pass it the log to search for and the directory to scan. If fuzzy searching was enabled, we supply the recurse_files() method an additional optional argument by setting logic to "equal". Without specifying this optional argument, the function will check whether the log is a substring of a given file rather than an exact match. We store any resulting hits in the event_log variable.
def main(evidence, image_type, log, win_event, fuzzy):
# Create TSK object and query event log directory for Windows XP
tsk_util = TSKUtil(evidence, image_type)
event_dir = tsk_util.query_directory(win_event)
if event_dir is not None:
if fuzzy is True:
event_log = tsk_util.recurse_files(log, path=win_event)
else:
event_log = tsk_util.recurse_files(
log, path=win_event, logic="equal")
If we do have hits for the log, we set up the event_data list, which will hold the parsed event log data. Next, we begin iterating through each discovered event log. For each hit, we extract its file object, which is the second index of the tuple returned by the recurse_files() method, and send that to be temporarily written to the host filesystem with the write_file() method. This will be a common practice in further recipes so that these third-party libraries can more easily interact with the file.
if event_log is not None:
event_data = []
for hit in event_log:
event_file = hit[2]
temp_evt = write_file(event_file)
The write_file() method is rather simplistic. All it does is open a Python File object in "w" mode with the same name and write the entire contents of the input file to the current working directory. We return the name of this output file back to the main() method.
def write_file(event_file):
with open(event_file.info.name.name, "w") as outfile:
outfile.write(event_file.read_random(0, event_file.info.meta.size))
return event_file.info.name.name
Back in the main() method, we use the pyevt.check_file_signature() method to check whether the file we just cached is a valid evt file. If it is, we use the pyevt.open() method to create our evt object. After printing a status message to the console, we iterate through all of the records within the event log. The record can have a number of strings, and so we iterate through those and ensure they are added to the strings variable. We then append a number of event log attributes to the event_data list, including the computer name, the SID, the creation and written time, the category, source name, event ID, event type, the strings, and the file path.
You may notice the empty string added as the second-to-last item in the list. This empty string is there due to a lack of an equivalent counterpart found in .evtx files and is necessary to maintain proper spacing as the output spreadsheet is designed to accommodate both .evt and .evtx results. That's all we need to do to process the legacy event log format. Let's now move on to the scenario where the log file is an .evtx file.
if pyevt.check_file_signature(temp_evt):
evt_log = pyevt.open(temp_evt)
print("[+] Identified {} records in {}".format(
evt_log.number_of_records, temp_evt))
for i, record in enumerate(evt_log.records):
strings = ""
for s in record.strings:
if s is not None:
strings += s + " "
event_data.append([
i, hit[0], record.computer_name,
record.user_security_identifier,
record.creation_time, record.written_time,
record.event_category, record.source_name,
record.event_identifier, record.event_type,
strings, "",
os.path.join(win_event, hit[1].lstrip("//"))
])
Thankfully, both pyevt and pyevtx libraries handle similarly. We start by validating the file signature of the log search hit using the pyevtx.check_file_signature() method. As with its pyevt counterpart, this method returns a Boolean True or False depending on the results of the file signature check. If the file's signature checks out, we use the pyevtx.open() method to create an evtx object, write a status message to the console, and begin iterating through the records present in the event log:
After storing all strings into the strings variable, we append a number of event log record attributes to the event log list. These include the computer name, SID, written time, event level, source, event ID, strings, any XML strings, and the event log path. Note there are a number of empty strings, which are present to maintain spacing and fill gaps where an .evt equivalent is not fount. For example, there is no creation_time timestamp as seen in the legacy .evt logs, and therefore, an empty string replaced it instead.
elif pyevtx.check_file_signature(temp_evt):
evtx_log = pyevtx.open(temp_evt)
print("[+] Identified {} records in {}".format(
evtx_log.number_of_records, temp_evt))
for i, record in enumerate(evtx_log.records):
strings = ""
for s in record.strings:
if s is not None:
strings += s + " "
event_data.append([
i, hit[0], record.computer_name,
record.user_security_identifier, "",
record.written_time, record.event_level,
record.source_name, record.event_identifier,
"", strings, record.xml_string,
os.path.join(win_event, hit[1].lstrip("//"))
])
If the given log hit from the search cannot be validated as either a .evt or .evtx log, we print a status message to the console, remove the cached file with the os.remove() method, and continue onto the next hit. Note that we only remove cached event logs if they could not be validated. Otherwise, we leave them in the current working directory so as to allow the user the opportunity to process them further with other tools if desired. After we have finished processing all of the event logs, we write the parsed list of lists to a CSV with the write_output() method. The two remaining else statements handle situations where there are either no event log hits from our search or the directory we scanned for does not exist in the evidence file:
else:
print("[-] {} not a valid event log. Removing temp "
"file...".format(temp_evt))
os.remove(temp_evt)
continue
write_output(event_data)
else:
print("[-] {} Event log not found in {} directory".format(
log, win_event))
sys.exit(3)
else:
print("[-] Win XP Event Log Directory {} not found".format(
win_event))
sys.exit(2)
The write_output() method behaves similarly to that discussed in the previous recipe. We create a CSV in the current working directory and write all of the parsed results to it using the writerows() method.
def write_output(data):
output_name = "parsed_event_logs.csv"
print("[+] Writing {} to current working directory: {}".format(
output_name, os.getcwd()))
with open(output_name, "wb") as outfile:
writer = csv.writer(outfile)
writer.writerow([
"Index", "File name", "Computer Name", "SID",
"Event Create Date", "Event Written Date",
"Event Category/Level", "Event Source", "Event ID",
"Event Type", "Data", "XML Data", "File Path"
])
writer.writerows(data)
The following screenshot shows basic information about events in the specified log files:
The second screenshot shows additional columns for these rows:
