Some of the most common complaints cyber security officers make are that management doesn’t support them and—as the famous comedian Rodney Dangerfield is known for saying—“I get no respect.” Another complaint is that the costs and benefits of a cyber security program cannot be measured.
As for the first two, you get support, because you are being paid—and these days, more often than not, quite handsomely—and you have a budget that could have been part of corporate profits. Furthermore, respect is earned. Besides, if you want to be popular, you are definitely in the wrong profession.
That last question often comes right after a successful denial-of-service attack or some other attack on the corporate systems or Web sites. Of course, many cyber security officers respond by saying that it can’t be measured. That is often said out of the cyber security officer’s ignorance of processes to measure costs or because the cyber security officer is too lazy to track costs.
The more difficult question to answer is, “What are the measurable benefits of a cyber security program and the functions that provide support under the cyber security program?” Of course, one could always use the well-worn-statement, “It can be measured only as a success or failure depending on whether or not there have been successful attacks against our systems.” The truth is that many attacks go unnoticed, unreported by the users or information technology (IT) people. Furthermore, separating attacks from “accidents” (human error) is usually not easy; however, metrics can help in the analyses.
What Is Cyber Security Program Metrics Management?
Cyber security program metrics management is the managing of a cyber security program and related functions through the use of metrics. It can be used where managerial tasks must be supported for such purposes as backing the cyber security officer’s position on budget matters, justifying the cost-effectiveness of decisions, or determining the impact of downsizing on providing cyber security program service and support to customers.
The primary process to collect metrics is as follows:
• Identify each cyber security program function;
• Determine what drives that function, such as labor (number of people or hours used), policies, procedures, and systems; and
• Establish a metrics collection process. The collection process may be as simple as filling out a log for later summarization and analysis. The use of a spreadsheet that can automatically incorporate cyber security program statistics into graphs is the preferred method. This will make it easier for the cyber security officer to use the metrics for supporting management decisions, briefings, etc.
The decision to establish a process to collect statistics relative to a particular cyber security program function should be made by answering the following questions:
• Why should these statistics be collected?
• What specific statistics will be collected?
• How will these statistics be collected?
• When will these statistics be collected?
• Who will collect these statistics?
• Where (at what point in the function’s process) will these statistics be collected?
By answering these questions for each proposed metric, the cyber security officer can better analyze whether a metrics collection process should be established for a particular function. This thought process will be useful in helping explain it to the cyber security program staff or management, if necessary. It will also help the cyber security officer decide whether he or she should continue maintaining that metric after a specific period of time. Since the corporate cyber security officer had begun with an analysis of cyber security program requirements (drivers) that led to the identification of a cyber security officer charter that led to the identification of cyber security program functions with process flowcharts, the task of developing metrics will be much easier. That is because each step noted in the cyber security program functions’ flowcharts can be a point of quantifying and qualifying costs of performing each specific function.
All metrics should be reviewed, evaluated, and reconsidered for continuation at the end of each year, or sooner—when a requirement changes, a function may also change. Remember that although the collection of the metrics information will help the cyber security officer better manage the cyber security program duties and responsibilities, a resource cost is incurred in the collection and maintenance of these metrics. These resources include:
• People who collect, input, process, print, and maintain the metrics for you;
• Time to collect, analyze, and disseminate the information; and
• The hardware and software used to support that effort.
When using these metrics charts for management briefings, one must remember that the chart format and colors are sometimes dictated by management; however, which type of chart is best for analysis or presentation to management is probably up to the cyber security officer.
The cyber security officer should experiment with various types of line, bar, and pie charts. The charts should be kept simple and easy to understand. Remember the old saying, “A picture is worth a thousand words.” The charts should need very little verbal explanation.
If the cyber security officer will use the charts for briefings, the briefing should comment only on the various trends. The reason for this is to clearly
and concisely present the material and not get bogged down in details, which detract from the objective of the charts.
One way to determine whether the message of the charts is clear is to have someone look at each chart and describe what it tells him or her. If it is what the chart is supposed to portray, then no changes are needed. If not, the cyber security officer should then ask the viewer what the chart does seem to represent and what leads him or her to that conclusion. The cyber security officer must then go back to the chart and rework it until the message is clear and is exactly what the cyber security officer wants the chart to show. Each chart should have only one specific objective, and the cyber security officer should be able to state that objective in one sentence, such as “This chart’s objective is to show that cyber security program support to corporate is being maintained without additional budget although the workload has increased 13%.”
The following paragraphs identify some basic examples of cyber security program metrics that can be collected to assist a cyber security officer in managing a cyber security program and briefing the management on the program and the program’s organization. By the way, when establishing a briefing to management in which the metrics charts will be used, a similar chart can be used to start off the briefing. That chart tracks the requirements (drivers) that can be traced to each function. One may also want to provide more detailed charts tracking specific requirements to specific functions.
Of course, as the cyber security officer, you would want to get more specific and track to a more detailed level of granularity. In fact, the cyber security program staff responsible for leading a specific function should be tasked with developing this chart or charts. That way, the staff will know exactly why they are doing what they do. The next step would be for them to track their workflow, analyze it, and find more efficient ways to do the job. At the same time they would also look at current costs and cost savings as more efficient ways are found to successfully accomplish their jobs.
The cyber security officer must remember that metrics are a tool to support many of the cyber security officer’s decisions and actions; however, they are not perfect. Therefore, the cyber security officer must make some assumptions relative to the statistical data to be collected. That’s fine. The cyber security officer must remember that metrics are not rocket science, only a tool to help the cyber security officer take better-informed actions and make better-informed decisions. So, the cyber security officer
should never get carried away with the hunt for “perfect statistics,” or become so involved in metrics data collection that “paralysis by analysis” takes place.
The spreadsheets and graphs used for metrics management can become very complicated, with links to other spreadsheets, elaborate three-dimensional graphics, etc. That may work for some, but the cyber security officer should consider the KISS (keep it simple, stupid) principle when collecting and maintaining metrics. This is especially true if the cyber security officer is just getting started and has no or very little experience with metrics. One may find that the project leads who are developing an “automated statistical collection” application are expending more hours developing the application—which never seems to work quite right—than it would take to manually collect and calculate the statistical information.
It is also important, from a managerial viewpoint, that all charts, statistics, and spreadsheets be done in a standard format. This is necessary so that they can be ready at all times for reviews and briefings to upper management. This standard is indicative of a professional organization and one that is operating as a focused team.
Cyber security officers who are new to the cyber security officer position, or management in general, may think that this is somewhat ridiculous. After all, what difference does it make as long as the information is as accurate as possible and provides the necessary information? This may be correct, but in the business environment, standards, consistency, and indications of teaming are always a concern of management. Your charts are indicative of those things.
The cyber security officer has a hard enough job getting and maintaining management support. The job should not be made more difficult than it has to be.
Another negative impact of nonconformance of format will be that the attendees will discuss the charts and not the information on them. Once “nonconformance to briefing charts standards” is discussed, management has already formed a negative bias. Thus, anything presented will make it more difficult to get the point across, gain the decision desired, and meet the established objective of the briefing.
It is better just to follow the established standards than to argue their validity. It is better to save energy for arguing for those things that are more important. After all, one can’t win, and the cyber security officer does not want to be seen as “a non-team player” more than necessary.
Of course the number, type, collection methods, etc., that the cyber security officer will use will be dependent on the environment and the cyber security officer’s ability to cost-effectively collect and maintain the metrics.