CHAPTER 15
Metrics—Overall Control
We start this book’s Controlling Processes part with this chapter on metrics. To effectively control the maintenance effort, the manager must really know what is going on. Of course, some level of control can be exerted on the maintenance effort without metrics. There will be anecdotal events that dictate what needs to be addressed, and the manager can assume total knowledge of everything that is happening is available. But that is just an illusion.
There are three kinds of people in the world: those who “know what happened,” those who “think they know what happened,” and those who say, “What happened?” People who think they know what happened are less effective as leaders than the people who know what happened. Metrics are what makes the difference. We won’t even talk about the “What happened?” people.
Single-engine aircraft pilots can fly across the country through clouds and rain using only the plane’s instruments and avionics. It is an exhilarating experience, after flying for several hours without seeing anything outside the cockpit, to suddenly break out of the clouds at only 200 feet above the ground with beckoning runway lights in front of you. For maintenance, metrics provide the same indicators as the aircraft’s instruments and avionics—they permit a safe arrival at your destination.
Maintenance/Project Difference
One of the differences between maintenance and projects is in the metrics for measuring success. The metrics for projects focus on the deliverables, when they’re completed, and the cost of producing them. The metrics are straightforward. Project management uses metrics such as earned value to determine if the project’s deliverables are on track relative to expected cost and schedule. The customer wants the answer to the question, “As of today, is the project going to deliver what I need on schedule and on budget?”
The metrics for maintenance, on the other hand, focus on the quality of the service, its timeliness, and its cost. For maintenance, the customer asks a different question: “As of today, has the maintenance team delivered the system availability specified in the Service Level Agreement (SLA) and satisfied the needs of our users while providing good value for the cost I am paying?” The manager needs to be able to answer this question on an ongoing basis.
Appropriate Metrics
At this point you may be thinking that we are advocating measuring everything. This is not the case, due to the cost. We have to ask whether each metric is worth the effort it takes to produce it. Some items may not have enough value to justify the cost of measuring them. On the other hand, there is a saying that you don’t get what you don’t measure. The metrics must therefore be balanced.
The best approach to metrics has two levels. Only the highest level will be reported outside the group. The highest level should be business outcome metrics that are mutually agreed to by the business customer and the maintenance team. These metrics should be clearly documented in the SLA.
Figure 15-1: Metrics Table From SLA
Figure 15-1 shows a sample table from an SLA. Each metric is defined and includes the committed target value to attain, the reporting frequency, and the hours that the business needs the system to be operational. For some businesses, the hours will be 8 a.m. to 6 p.m. Monday through Friday; for others, the business needs the system to be operational around the clock.
The SLA should have defined all the metrics that the customer wanted, but was the customer articulate enough at the time the SLA was written to capture all its needs? There may indeed be other needs, so you should contact the customer and review what metrics are important to it, starting with the SLA list.
The lower-level metrics should be the early-warning metrics that will indicate if any one of the highest-level metrics might be at risk of something going off track. The lower-level metrics can be the same measures with a target value that will be encountered well before the SLA metrics target is reached. As an example, the SLA target for availability could be 95 percent, but the lower level could be set at 97 percent. Each metric should have a clear description of how it will be measured along with the frequency at which the measurement will be taken.
The following are possible metrics to track per review period (typically monthly) and to include in a metrics report for IT management and the business:
Cost:
• Budget vs. Actual
• System, Subsystem, and System Availability
• Hours of Rework to Correct a Production Change (check on quality of testing)
Customer Satisfaction:
• Trend the Customer Satisfaction scope. Include the overall scope and individual questions scope. Chapter 12, “Customer Care,” provides more information.
Turnaround Time:
• Average Time to Close an Incident for Each Severity Level
• Age of Open Incidents for Each Severity Level: maximum, average
Service Throughput from Work Tracking Tool:
• See Figure 15-2.
Defects and Production Incidents are a significant variable in maintenance, so we break up the metrics by severity levels. The severity levels in Figure 15-3 should be listed in the SLA (as shown in Figure 5-1, the template for the SLA). These severity levels must be based on business impact, not on user opinions. For your situation, there may be other logical breakouts you may want to provide for better clarity, such as listing metrics separately for each geographical site.
Reporting Metrics
Tracking and reporting metrics that the customer cares about is important for developing and maintaining trust between you and the customer. The more transparent your maintenance team’s operation is to customers, the more comfortable they will be. You may receive some initial tough, challenging questions, but that is a good thing.
Figure 15-2: Monthly Metrics Template From Work Tracking Tool
The development of the Communication Plan should define the frequency of reporting the metrics and specify to whom they will be reported. Not all customer groups will want to see all the metrics, so the Communication Plan should spell out exactly what the customer wants. Always take the perspective of customers when deciding what metrics to show them.
Practical metrics can go a long way in justifying the SLA effort estimated and the team size. They can even point out the faulty assumptions of customers and the maintenance manager. Don’t rely on customers to remember everything you did for them; provide the data that paints the picture for them.
As stated in Chapter 11, “Work Tracking,” an effective way to report metrics from the Work Tracking Tool is to provide your customers with web browser access to the information. Some simple web screens summarizing the records stored in the Work Tracking Tool can go a long way in providing the amount of information customers would like to see.
For extra value to customers, you can provide additional screens with more detail than that contained in the summary screens. This same web portal can also provide access for customers to actually enter new request tickets.