CHAPTER 8: MEASURE, MONITOR, AND REVIEW

A useful information security management system (ISMS) is one that helps an organization achieve its information security objectives. Those objectives should be linked to its business, regulatory, and contractual obligations, and should be delegated to appropriate levels within the organization.

ISO 27001 requires the organization “to continually improve the suitability, adequacy, and effectiveness of the ISMS.” The corrective action requirements of the Standard are met by an effective ISMS audit plan, competent review of nonconformities (part of the responsibility of the information security manager), the incident response procedures, and the related documentation.

The combination of effective monitoring, measuring, and corrective action processes—together with a formal review process and strong internal audit structure—within the context of an ISMS will enable an organization to start driving continual improvement throughout the organization. (Of course, if the original approach to the ISMS implementation was to structure it as a continual security improvement project, then this concept will already be built into the underlying logic of the ISMS.)

A long-term approach to continual improvement should include measuring the effectiveness of the ISMS and of the processes and controls that have been adopted. ISO 27001 requires effectiveness measurements, and for the results from these to be included in the input to the management review meeting. Clearly, information security as an organizational function needs to be measured against performance targets in just the same way as other parts of the organization. The organization also needs to be able to measure progress toward its corporate security objectives, and this also is a requirement of ISO 27001.

In order to develop a useful set of metrics, an organization will have to identify what to measure, how to measure it, and when to measure it.

Key areas that should be considered for their contribution to the organization’s ISMS goals and key objectives should include:

• Effectiveness of identified controls and groups of controls that relate to the most significant risks identified in the risk assessment.
• Effectiveness and cost-effectiveness of the organization’s information security awareness, education, and training.
• Extent and effectiveness of vulnerability patching and management.
• Improvement in efficiency generated by access controls and external contracts.
• Effectiveness of the incident handling process.
• Effectiveness of perimeter security and speed of remediation through penetration testing.

Internal audit and testing

ISO 27001 requires organizations to conduct internal audits of the ISMS at planned intervals.

Your ISMS has to work in the real world. You identified risks, you deployed what appear to be appropriate controls, and you want to be sure of two things: first, that the controls work as intended, and second, that when they are overwhelmed (as they will be sooner or later) your emergency counter-measures also work. Your management system, including each and every control, is planned and deployed. The management system and every control are then tested to see if they work according to plan, and the management system and every control are improved in light of that testing.

There are four types of testing that should be considered. The first is internal audit, which involves a trained ISMS auditor following a documented procedure and asking for evidence that what is described in the procedure is what actually happens. As part of your ISMS project, you will need to put a team of trained ISMS internal auditors in place. These people can be drawn from around the business, appropriately trained, and—provided you ensure they never audit any part of the business for which they or their managers are responsible—they will meet your long-term audit team requirements.

The second is a limited ‘paper test.’ This is an intellectual exercise; it requires more than one person, and also requires familiarity with the vulnerabilities in the asset, the mechanisms of the control, and the mechanisms and makeup of the likely threats. Given this knowledge—that should be both current and experientially and technically based—the effectiveness of controls (such as incident management or business continuity controls) can be logically tested.

The third is a limited, real-life test. This could involve powering down the server room during normal operations to find out whether the APS systems and server shut-down procedures all work as specified, for instance. Real-life tests should not be carried out without first having taken extensive steps to ensure that if something does not work as planned the system can be restored to the point it was at when the test was executed. This type of testing includes penetration testing, which should be carried out by a specialist penetration testing firm and should test both your selected controls and your risk assessment: in other words, you should instruct your penetration tester to try to penetrate your system by methods you haven’t identified. You can assess later whether these are threats against which you need to control.

The fourth and final type of test is a large-scale scenario test, most usually used to test major cyber incident and business continuity plans. These tests usually try to telescope the events of several days into a much shorter space of time and require all those who would have roles in the real-life disaster to attempt to perform the required tasks in the role play. These tests require considerable planning and, again, it is a sensible area in which to deploy external, specialist expertise.

You will want to schedule audits and tests so all aspects of your ISMS are covered in the course of a year. You should do this on the basis that some controls need to be tested more regularly than others—carry out a risk assessment to determine the frequency of testing you will require. Your external certification auditors will want to see evidence of your internal audit and testing, the results of this activity, and details of how you used the findings of this activity to improve and tighten your ISMS. You should assume your external certification auditor will want to see evidence of at least one cycle of audits and tests. If you want to achieve certification after less than one year’s worth of testing, you will need to design a test and audit cycle that covers all the mission-critical aspects of your ISMS within a much shorter timeframe. This is not an unusual approach and most certification bodies accept there are a number of items that do not need to be tested that regularly.

Management review

Top management should review the performance of the ISMS on at least an annual basis. Inputs to the management review process will include all the results from internal audits and testing, as well as continual improvement activity and analysis of nonconformances and incidents that have occurred during the preceding period. As I said elsewhere, root cause analysis is the preferred approach.

The management review should look inwards at the performance of the management system, and at the metrics that describe how the ISMS is performing in relation to its objectives. It should also look outwards at the world in which the organization operates, to ensure it is taking appropriate steps in the context of changes to its operating and risk environment.