File integrity monitoring (FIM) is one way to detect changes to a known filesystem's files, and in the case of Windows, the registry. Typically, when a system has malicious activity, either changes are made to existing files or harmful files are placed in critical areas of the filesystem. In order to detect these changes, FIM tools create a hash database of the known good versions of files in each filesystem location. The tool can then periodically or real-time scan the filesystem looking for any changes to the installation including known files and directories. Hashing is used because any variation in the file will result in a different hash value, and therefore confirm there has been a change to the file, directory, or registry. The tool will then create an event that will need to be reviewed to ensure the detected addition, removal, or modification was expected. If yes, then the reviewer can comment and accept the new hash as the new baseline. Any subsequent scan for changes will use the newly accepted artifact version as the baseline. If the change was not expected, the reviewer can investigate to determine if the source of the change was malicious or an undocumented and unapproved change by a system administrator. Some tools in this space can rollback a detected change if malicious or unapproved by an internal change management process.
An example of calculating a hash for a file is shown in the following screenshot. I have created a file and entered text, then calculated the MD5 hash with the MD5 tool using the md5 test.txt command at the command prompt. This tool is native to OSX and Linux, but may need to be installed in Windows.
I then added more text modifying the file itself so a new hash would be generated indicating the modification of the original file. You will notice that the next screenshot has a different MD5 hash for the same file because it was modified. Note that changing the filename does not affect the calculated MD5 sum; the tool only detected content changes. More complex FIM tools have the option of detecting several attributes including timestamps, name, content, and permissions.
It is plausible to assume that this type of tool would need significant tuning to reduce false positives for actions such as a login to Windows. Typically, files and directories that change frequently should not be monitored to reduce tool output. An example of a file type that is not optimal to monitor is a log file. Log files are typically written to on a frequent basis and the hash would also change causing a flag in the FIM tool. Trial and error may be used to determine the areas of the filesystem that are prone to frequent and expected changes and can be set to be ignored by the tool. Exercise some caution, however, when doing this. For example, when using Metasploit (hacking tool) persistence in Windows there are two areas affected, the registry autorun key and the Windows temp directory. It is common to ignore these areas and malicious hackers are aware of this, so this is where changes are commonly made, hoping they will not be noticed, while a foothold on the system is established. Some tools will have these areas monitored but will set them to auto accept and are available for forensic purposes. The primary use case for FIM is enforcing review of changes to business critical systems and in the case of PCI DSS to enforce a review of system changes where credit card numbers are stored, processed, and transmitted.
A caveat to using this type of tool is the accidental addition of malware or unapproved configuration added to the system baseline. This renders the protection ineffective because now the baseline is tainted and the malware may go undetected. This is also true for unapproved configurations that may be harmful to the security posture of the system. FIM is not only a security tool, but is required by PCI DSS and can determine if a system is compliant to the standard. Some FIM solutions have the capability to run checks and provide reporting for configuration compliance to PCI, SOX, CIS, and other standards, which can be a compliance benefit to running the tool.
Because this type of tool is used heavily for compliance, it is often implemented widespread without any emphasis on the critical systems, and too much data is generated to be actionable. Another challenge is that on some operating systems, a simple action such as changing permissions on a directory or adding a user to a group, will generate a very cryptic output that is not humanly understandable or actionable. Filtering out these instances, yet capturing meaningful data, is a very intensive exercise that will take time to tweak and get buy-in from system owners who are responsible for reviewing the output. It may be possible to reduce system controls through the use of FIM reducing both the capital and operational expense of securing enterprise systems.
FIM is an excellent tool to detect changes to the filesystem including installed applications. This can be ideal in scenarios that require an application folder to be monitored to make sure the application is not rooted or manipulated in some way. As stated before, the challenge to getting value out of this type of tool is the rate of change a system may go through for standard operation. This may cause a significant number of non-event alerts that will need to be investigated and deemed OK and ignored in further scans, or place a threshold to reduce alerts. It is important to think like a malicious person would and double check your logic before disabling every noisy item in FIM; this could lead to undetected compromise of a system.
The general architecture of FIM solutions is a console and an agent deployment to provide detection and policy checking. The agent can typically run in two modes, real-time and manual. In real-time mode, the local agent is constantly looking for add, delete, and modification actions on the system and will report findings to the console in near real-time. The manual mode will sit idle and when the console initiates the scan of the system, the local agent will run and report findings since the last run. Both solutions have their advantages and disadvantages. The basic architecture is depicted in the following diagram:
Using FIM in real-time mode has several advantages, but the constant running of the tool may be taxing to a system that is loaded with several agents for various purposes. The primary advantage to real-time mode is exactly this; all add, delete, and modification actions are detected in real time allowing for almost immediate ability to review and remediate. However, this capability must be carefully weighed to ensure the changes can be reviewed in a timely manner and that the rate of change is not so high that the alerts are overwhelming. The ability to detect actions faster can be of great benefit in the instance there is malicious action, reducing a possible malware persistence or lateral intrusion. This method will require constant monitoring or the use of another technology such as a SIEM solution to provide some intelligence, threshold, and meaningful notification of the detected changes.
FIM in a manual mode configuration is the least taxing on the system because the scans only run when the console initiates the scan either adhoc or on a schedule. The benefit with this implementation is if scans are scheduled, then IT knows when the system may have higher memory and processor utilization and it ideally will not affect business operations. This method also provides an alert dump at the scheduled time versus spontaneously throughout the day as actions are detected. The internal processes used to process FIM output will determine the efficiency of this method. A caveat to this solution is that changes can go undetected for longer periods of time depending on how often scans are run on schedule. The organization will need to ensure that this method is within the accepted risk level and that this process is operationalized enough to meet the intended purpose of FIM. Much like how often anti-virus signatures are updated, the frequency will affect the efficiency.