Measuring the effectiveness of security policies is essential to maintaining leadership support. The quality control function is a good place to obtain these measurements. Typically, a quality control function will sample security controls and test their effectiveness. These measurements are often shared with leadership, stakeholders, control partners, and regulators. They are a good indication of the health of the system and level of adherence to information security policies within it.

To ensure accountability, you need to measure if employees are following the policies. Selecting the right performance measurement can be tricky. A common pitfall is to measure success as a percentage of implemented policy coverage. It’s easier to demonstrate the value security policies bring to the business when the business sees its operational risk being reduced. Therefore, the best measurement of whether employees are following policies is the actual reduction in risk that occurs.

For example, let’s say you have a security policy that requires all servers to be patched. More precisely, all critical security patches must be applied within so many days of their release. For this discussion, assume you know what a critical security patch is and can measure when it’s applied. Reporting that 90 percent of your servers have received the patch may sound good. But how much of the risk has been reduced? If 80 percent of your business runs through the 10 percent that has not been patched, your business is very much at risk. When measuring performance or effectiveness of policies, always ask, “How much actual risk to the business has been reduced?”

Measuring effectiveness is easier than measuring accountability. As in the prior example, the organization can quickly determine if the level of patch management is compliant with security policy. But suppose it is not. Who is accountable? It is often much harder to measure the percentage of employees that either follow or fail to follow policy. One method is to use effectiveness measurements to identify areas of high or low effectiveness. Then analyzing why certain areas were successes while other areas were failures leads you to accountability.

You can get a basic understanding if individuals are being held accountable for adherence to security policies by examining policy violations, incidents, and security awareness. These basic measurements are as follows:

• Number of security violations by employees reported—You should investigate any unexplained increase in this number to determine why an abnormal number of security violations occurred. One reason could be lack of training.
• Number of incidents that could have been avoided—When a security breach occurs, you need to determine the root cause. This root cause can tell you if a contributing factor was a policy failure.
• Completion and competency rate for security awareness—You should track which individuals have completed security awareness training. Additionally, the training should measure the level of competency with the material. This training needs to be refreshed and redone at least annually.