Tools can be (and often are) confused with resources. I’ll use a simple ­delineation between the two. Tools are items that can be used to do—to actually design, create, analyze, and publish metrics. An example is Microsoft Excel.

On the other hand, resources provide information that provides guidance or knowledge used for designing, creating, analyzing, and publishing metrics. Rather than an analysis tool (like Minitab), resources include textbooks on how to use software or perform statistical analysis, how-to videos, articles, blogs, books, and discussion groups (such as those found on networking sites like LinkedIn). There are also organizations (new and established) that you can join and participate in to learn more about metrics.

In this appendix, I’ll share some of my favorite tools and some that come highly recommended by colleagues and friends. Please don’t buy any of these tools on the basis of their inclusion here; instead, if something sounds good to you, research it further. Just as with metrics themselves—you’ll need to marry the possibilities to your specific requirements. Based on your root questions, your environment, and the programs you develop—different tools and resources may be called for.

Tools

Some tools play multiple roles, but most specialize in a primary function and have other functions as a supplement. Most trouble-ticket tracking tools work this way. They are designed (and do a good job at) capturing and tracking trouble tickets, and may also provide basic graphing tools. They don’t provide much in analysis; and provide nothing toward being a complete metric tool. But trouble-tracking tools are good at what they are supposed to do. The key is not to try to make a specific tool do more than it is designed to do.

The really good news is that a meaningful and useful metrics program doesn’t require a ton of statistical analysis or complicated charting. Always keep in mind that the purpose of a metrics program is to provide insights that can support decisions, direct investigations, and expose areas of concern.

Tools simply provide different ways of looking at the information you’ve gathered. Depending on your root question and what you are trying to learn from your metric, you can use many different forms of analysis. The tools I cover in this appendix are the few that I find useful and simple. They are simple for me to produce/use and simple for my audience to understand. Always remember your audience when displaying your metrics. Even if you use more complex analysis to get to your answers, you may need to find ways to display them in more simplistic terms.

Microsoft Excel

I’ve found Excel to be much more than spreadsheet software. I guess it was originally created to assist with accounting or bookkeeping. Ages ago, I used to explain that spreadsheet programs were good for working on numbers. If you wanted to perform math on your data, spreadsheets were the way to go. Today, Microsoft Excel has grown to be much more than a glorified calculator. I use it for much of my metrics work. Like most current software applications, Excel has more capabilities than most users realize or will ever use. Besides the breadth of functionality, Excel also provides a lot of ­flexible power, as in macros (mini programs), PowerPivot, and PowerView. I ­collect, analyze, and produce graphs for most of my work in Excel. With Excel add-ins, I can also perform more complex analysis inside the spreadsheets. With Excel 2013, I can handle as many records as I need. I use Excel as the foundation for my work.

Excel has a statistical add-in that comes with the full version, but you have to “turn it on.” It doesn’t come with this functionality preset. Turning it on will give you some basic statistical tools like histograms and regression tests.

SigmaXL

SigmaXL is a tool that can be accessed through Excel. It creates a more intuitive set of menu items in its own tab than the statistical add-in that comes with Excel. I was especially happy to find that it had a BoxPlot tool (graph), which I was unable to find in any other add-in. I am truly impressed with the SigmaXL capabilities.

Minitab

While working on my Green Belt in Six Sigma, I fell in love with SigmaXL. Before the honeymoon was over, my instructors told me about Minitab. They described it as better, smarter, easier to use, and more comprehensive than SigmaXL. The biggest problem with statistics is they’re just too hard to deal with, especially for the benefits gained. SigmaXL makes it worth the effort; and if Minitab is better (I haven’t used it yet), I want a copy.

Visualization Tools

Visualization tools are primarily (if not solely) designed to provide access to your data—in the form of dashboards, scorecards, or other visualizations. These can be used for metrics per my definition.

Theoris Vision Software

Theoris Vision Software provides a dashboard and reporting solution that includes charts, graphs, scorecards, maps, and ad-hoc reports. Everything is driven from the dashboards. I like that I can create my own visual and reporting content pretty easily. From what I’ve seen, the real power of this application lies in the unique ability it has to map to various data sources and files, instead of the traditional approach of pulling everything together into a spreadsheet or database.

Vision is not a metric design tool per se, but it is a powerful tool for visualizing your measures. It also allows others (stakeholders) to access your metrics on their own. I especially like the ability to set up hierarchies in the data, which in turn allows viewers to drill down into their data further. This capability is further expanded to allow the creation of ad-hoc reports at different levels of information. This can be a bad thing—depending on the level of maturity of your audience—but for the most part, it’s pretty slick. Overall, this is really an impressive tool for displaying metrics and starting those critical conversations. Currently, Vision is mostly focused on the health industry, but the competition had better watch out as Vision branches into other industries. A really nice tool which I’ve added to my wish list.

iDashboards

iDashboards is exactly what you’d imagine by its name: an excellent tool for creating and displaying dashboards—or compilations of Key Process Indicators (what I’d call measures). With iDashboards, you could use your measures to create a view of your metric for a given question.

I like both Vision and iDashboards for this purpose. The question for me then becomes cost and ease of use. If you are thinking of obtaining a tool for displaying your data in this manner, I’d encourage you to spend time demo-ing both products. This one is also on my wish list—and I’ll be happy with either as a surprise for Christmas.

Tableau

Tableau is not an add-in, but it works well with Excel. You can easily import data into Tableau from Excel or other common tools. Tableau’s power is in the ability to quickly and easily try different graphical representations (visualizations) of your data. I know some people who love it and use it before they do any thorough analysis. I haven’t found it works for me, but I do see the potential value (I have a copy, but haven’t been able to put it to a lot of use).

Survey Tools

The following three survey tools each offer different benefits. In order, the first provides some useful analysis of what’s important to your customers as well as how well you provide those services. The second is a third-party service (not really a tool in the true sense) and makes the survey work easy. The last is a favorite of mine as it allows you to create what you need, customizing your surveys to fit your requirements (and the cost is attractive).

TechQual+

TechQual+ (www.techqual.org) is a good example of a survey tool; it was created by my friend Timothy Chester, the CIO at the University of Georgia. Most of the questions are preset in TechQual+, causing some limitations; but these are also its strengths. Since the questions are standardized, you can compare your results to others who chose to use this tool. There are other third-party survey organizations (HDI, for example), but a large benefit of TechQual+ is that it is free.

HDI

The HDI Customer Satisfaction Index is a survey service. HDI does ­everything for you: they survey your customers, tabulate your results, and provide you with reports in multiple formats. They also provide comparisons to others (for example, by industry or all other customers). HDI is only one example; there are other third-party survey services that offer this service. I suggest you price shop and look for ones that already have a large ­customer base in your industry. Customer satisfaction surveying is an interesting ­business niche.

SurveyMonkey

I confess—I like SurveyMonkey; partly because it’s free (if you use it sparingly; though, if you want to use it on a larger scale, the costs are very reasonable), but mostly because of its simplicity. With SurveyMonkey, you build the survey; then you provide a link to the survey to your customers. SurveyMonkey also offers simple analysis tools, but I usually download the results into Excel and do my own analysis.

IT Solutions/Business Intelligence Tools

Many “IT solution” companies now include dashboards and scorecards in their service packages. This is a clear indicator of the need for metrics and the power of software to help deliver them. IT solutions packages may include the metrics tools as an add-in to the suite of services—pulling the data byproducts from key offerings (process control, management, architectural design, etc.).

The major difference I’ve found between these tools and the stand-alone tools is in the scope. If you don’t need (or can’t afford) a large-scale solution set, you can get a lot out of the tools specifically designed for metrics. If you’re looking at purchasing (or already own) a large-scale IT solution, you may want to look into its capabilities to also provide metrics assistance.

ASE 10

An example of a data-centric toolset for organizational improvement using scorecards, dashboards, and measurements is ASE 10, from ActiveStrategy (activestrategy.com). It’s a bit complex, but offers pricing based on company size. ASE 10 is heavily based on predefined methodologies, but seems to have enough flexibility to work with the metrics that you design. I haven’t used this tool, but it has been recommended by a colleague whose opinion I trust.

The issue for most larger-scale tools is that they may offer too much. Ignoring the cost, these tools offer more capability than most people need—especially if you are just starting on your metrics journey.

Other Tools

Other tools may not fit the definition of a metric tool at all, but be very helpful to your metric efforts. The two examples I offer are at different ends of a scope/size spectrum. QPR is a larger scale process improvement tool which has useful applications to a metrics effort. PowerPivot is a tool which works with (and “in”) Excel.

QPR

QPR (QPR.com) is an example of a business-driven solution. Its scope is so large that I can’t tell you about it all. QPR is used mostly by companies ­outside of the United States; but I believe it will make a big splash on our shores soon. Rather than a simple, lower cost, limited-use tool, QPR’s solution is a mid-range, enterprise-level solution.

QPR’s web-based solutions can be selected based upon your need. If you need to build an understanding of your processes, one of its tools, “Process Analyzer,” assists in developing business process diagrams using a logging file input structure. If you need to share your database of business processes, “ProcessDesigner” provides that solution. Most metric-centered reporting requirements can be satisfied with the “Metrics” solution. If you need to ­integrate business process reporting and metrics, a combination of these ­solutions provide you with an integrated management reporting system.

The cost will reflect its expansive power. I include it as an example of a ­high-end tool, and because it does so much more (process analysis, process management, etc.) than metrics, the higher costs are no surprise. I especially like that such a nice enterprise-level solution includes specific tools for metrics.

PowerPivot

Unlike most of the tools, PowerPivot isn’t an analytical tool at all. It allows you to use Excel to be more like its brother, Microsoft Access—a database tool. Although most metrics are number-based efforts, there are many times when it would be useful to have a relational view of the data. A relational database would be the perfect tool if it had the ability to do mathematical and statistical analysis on the data. PowerPivot promises to give you the best of both worlds—a number-based program you can treat as a relational database.

Resources

Whereas I told you to research before you buy any tools (tools are high-cost compared to a book or a membership in an organization), in this section, I recommend you try the resource. If you don’t want to buy the book, check it out from your local library. For the most part, to determine the usefulness of a resource, you’ll need to have complete access. Most of the resources I offer are either free or low-cost (less than $50).

When you search for resources, depending on your industry, you will find many to choose from. In the information technology arena, I’ve been hearing a lot about the COBIT framework, ISO/IEC 20000 (international standard for IT service management), and the Information Technology Infrastructure Library (ITIL).

Rather than provide you a list of web sites, I will share some ways to perform a search. Search engines offer results in different formats: video, web, images, blogs, shopping, etc. Most useful will be web, video, and blogs. The web classification is where you’ll find everything from articles in e-zines, books, and encyclopedic definitions, to how-to guides for developing metrics.

If you search “metrics” you will find too many results on the metric system of measurement. You’ll want to narrow your search. “Performance Metrics” will bring you a lot closer to what you’re looking for. Even then, you may want to narrow your search depending on your particular needs and industry. For example, you can search on “IT metrics” or “IT performance metrics” if you are in the information technology arena. You can also search on “business intelligence” (the newest catch phrase for data-based decision making) or “IT solutions.”

Depending on your industry, you may find a healthy store of standards, bench-marks, and predefined data, measures, and information for your metrics. The financial industry is one example of a robust metric environment. Another is the manufacturing industry. If you are reading this book, you are not likely in an industry that has an established metrics framework. Chances are you are in need of meaningful metrics for your organization and your processes. Even so, you can learn from other industries and their metrics. You may be able to leverage some existing works for your own metrics efforts.

A simple search via your favorite web-based search engine returns a long list of measurement, statistical analysis, and metrics tools. I won’t provide you with a list that you can visit on your own. Instead I offer insights and a short list of resources and references.

While I built the list from tools and references that I’ve personally used, I highly recommend that you do what I’ve preached from the beginning—investigate for yourself. I have found some books to be “on target” and others to have views that I would argue strongly against. Every book, blog, and article I’ve read has been useful in developing my overall view and concepts about metrics. Even the ones I’ve found outlandishly off-target have proven to be beneficial to the overall concept I offer in this book.

Don’t discard the entire work because you find some portions to be “wrong,” in your viewpoint. You can learn much from those who disagree with you. One of my colleagues who helped in the writing of this book disagreed with me more often than we agreed; it was one of the reasons that I asked him to be the technical reviewer for the book. I trusted him to provide an honest view, even if it were a totally dissenting one. While I believe in the concepts and tools I’ve presented, I’m open to other opinions. I welcome them as they should help to make my understanding of how to make metrics work better.

I want you to look at the resources and references listed here, and any others you investigate later, in the same way. There are no silver bullets, there is no holy grail. There is no one right way to do organizational development or process improvement. There is no one way to do metrics. Stay open to new ideas and different opinions. And always make sure what you use works for you. Don’t use it because I or anyone else say to, use it because you’ve tried it and it works for you.

So, let’s look at some of the resources and references I’ve found useful in my metric journey.

Web Sites

The following are web sites that I’ve found useful.

XPC Palladium Group

XPC (http://community.thepalladiumgroup.com) is primarily a community for discussing Balanced Scorecard methods, but I have found it a good place to converse on metrics in general. Most of the participants on the site are disciples of Kaplan and Norton and believe in using measurement in ways I disagree with. The good news is they are open to other opinions. It is a well-run web site. Just recently I have heard that they are going to charge for membership—and as you may have ascertained from my opinions on tools, I don’t believe in paying for the opportunity to network. As with all of my recommendations, check it out (especially if you have to pay) before you buy.

LinkedIn

I like networking; especially networking for professionals. In addition to the LISTSERVs I participate in through different organizations, I enjoy LinkedIn (www.linkedin.com). LinkedIn groups allow for conversations and discussions on pretty much any topic you want—and you can simply create a new group/topic if it doesn’t exist. I belong to more than one group concerned with metrics (Performance Measurement, IT Performance Measurement, and IT Metrics–CEITPS) and have found them to be very useful. And membership is free.

The Consortium for the Establishment of Information Technology Performance Measures (CEITPS)

CEITPS (www.ceitps.org) is a nonprofit organization that I founded for the sole purpose of developing standards for IT performance measures. It is a very young organization. All standards created by this organization will be made available free to the public via the web site. Membership has a minimal fee and the biggest benefit you get for your money is that you are given the opportunity to help in creating and voting on the standards. The membership income is used to pay for the web presence only.

smartKPIs

smartKPIs.com is a repository of Key Process Indicators (measures). Since it offers free access to a good portion of its KPIs, I think it is worthy of ­mention. I can’t recommend paying for any of their offerings (only premium subscribers have access to calculation, references, and PDF export and filtering functionalities) as it goes against my beliefs toward benchmarks and canned metrics. But, if you want to see what others have come up with, the free catalog of examples is a good place to start.

I won’t repeat all of the cautions I’ve offered in the book—but I will suggest that if you use this (or other comparable references) that you do so with a grain of salt and also ask around. Your industry peers should be great sources, and by asking them you’ll build your relationships and your professional network.

Books

These books helped me immensely when I was trying to get my head and arms around metrics. I think they will help you too.

How to Measure Anything

How to Measure Anything: Finding the Value of Intangibles in Business by Douglas W. Hubbard (John Wiley & Sons, 2007). I love Hubbard’s positive, can-do ­attitude when it comes to finding ways to measure literally anything. I especially enjoyed his work on calibrating your ability to estimate accurately. Some of it was a little too deep for me, but I found almost all of it useful and an easy read. All in all, it is a nice text on how to measure and estimate—untethered to any specific improvement methodology. I recommend this book, but suggest you may find yourself picking and choosing chapters to read or reference.

Transforming Performance Measurement

Transforming Performance Measurement: Rethinking the Way We Measure and Drive Organizational Success by Dr. Dean R. Spitzer (American Management Association, 2007) is another favorite. I found it easy to read (with minor exceptions). I consider Dean a kindred spirit. Most of my disagreements come in how to deal with the fear, uncertainty, and doubt that surround metrics. I find that we are on the same wavelength, however, when it comes to the problems and hurdles you have to overcome to make metrics work. I believe we go to the “same church, different pew.” Definitely worth the read.

The Intelligent Company

The Intelligent Company by Bernard Marr (John Wiley & Sons, 2010) is yet another favorite. I came upon it well into the writing of this book and knew that I had found another member of the metrics family. As with Dr. Spitzer’s book, I found a fair amount to argue against in Marr’s book. But I find this book more useful than one with too much jargon or technical speak. I learn a lot (more) from viewpoints different than my own. Overall there is more that I agree with than I disagree with and it’s an “easy” read. I recommend this book for your library.

Measuring What Matters

Measuring What Matters: Simplified Tools for Aligning Teams and Their Stakeholders, by Rod Napier & Rich McDaniel (Davies-Black Publishing, 2006). I found this book to be less a guide for developing metrics and more a manual for the American Society for Quality (ASQ). That’s not a bad thing, but it wasn’t as much help with developing metrics as I would have liked, especially based on the title. I like it much more as a general organizational development book than a metrics book. It’s worth a perusal.

Why Organizations Struggle So Hard to Improve So Little

If you’re interested in organizational development, I humbly include Why Organizations Struggle So Hard to Improve So Little: Overcoming Organizational Immaturity, by Michael Langthorne, Donald Padgett, and me. I’ve actually read it twice since it was published in 2010 by Greenwood. It is a very easy read with important insights to why you may be struggling to improve or change your organization. The chapter on metrics makes a good introduction to this book. If you’re looking at implementing organizational change, I recommend you read it.

Other Books

There are books that I wouldn’t recommend for the purpose of developing a metrics program, but are useful in performing analysis; and if you have room on your library shelves, it wouldn’t hurt to include them. One that I like in particular is IT Measurement: Practical Advice from the Experts (Addison-Wesley, 2002) a compilation by the International Function Point Users Group. I have a special place in my metrics heart for this group since my first metrics mentor, Errol Shim, was a past president of the national group. The 43-chapter book was written by a variety of experts. Some definite gems can be mined here—and hopefully I’ve given you the tools necessary to find the gems that fit your needs.

Recap

Tools are useful for performing the work—designing, creating, analyzing, and publishing metrics. Resources are references that make doing the work easier or better. Unfortunately, tools can be expensive. The good news is that the methods I’ve offered for developing a metrics program don’t require any particular tools. You can do quite well with whatever tools you have already available. But as you become proficient at metrics development, you may want more out of your toolset, so I offered some tools to consider or explore. Remember, it’s only a starting point—find what will work best to meet your requirements and your budget.

Resources, on the other hand, should be investigated as early as possible. I’d be flattered if you only used this book: dog-eared it, highlighted the best passages, wrote in the margins, and used it to help you develop your metrics program. But, chances are you won’t agree with everything I’ve offered. Or you may want confirmation through other works. Or you may feel that I left some gaps in your comprehension of the material. I encourage you to read other books, articles, and papers on the topic.

I have faith in what I’ve been teaching on metrics and I welcome arguments to the contrary. If the concepts within this book are correct, then they will stand up to thorough scrutiny. To that end, feel free to contact me and offer your opinions—be they in agreement or disagreement. Join me on LinkedIn ­discussion groups, send me an e-mail, or post your thoughts, questions, or arguments on my web site. The bottom line is simple: do something! Learn more, try more, do more. Share your opinions, try the suggestions I’ve offered, create a meaningful metric and see how it goes.

Now Comes the Hard Part

The hard part is putting any or all of this into practice.

I consider one of my first mentors in metrics, Erroll Shim, a giant of a man. I consider him a giant in his field, function point analysis and metric analysis. He taught me a great deal and he helped set me on the path that led to the development of much of what you’ve read here. His expertise was very impressive—he would accurately predict the complexity of a software change and estimated correctly the effort and time required (which were normally very different than our unit had estimated).

But, the problem was that his abilities were his own. They weren’t transferrable. It was borne of years of experience. I wanted to develop a simplified method to provide the types of insights he produced independently, to anyone who needed it. These insights were at once more general in nature (they deal with metrics across the business spectrum) and specific in the methodology—building from a root question, using data, measures, information, and other metrics to tell a complete story.

I want you to develop meaningful metrics. I want you to be able to do so without obtaining a certificate in statistics, buying expensive tools, or spending months in training. I believe I’ve given you the knowledge to make a good start. But you’ll need more than a good effort.

Unfortunately, even if you follow the guidance in this book, you will encounter more than a fair share of resistance. It would be nice if you could introduce any of the concepts I’ve offered—from root questions, to the metrics framework and taxonomy, to the use of documentation, how to use and not use metrics, the Answer Key, triangulation, starting with Effectiveness—without having to fight for their acceptance.

When I attempted to create a metrics program for my own organization, I ran into a lot of resistance. I believed it stemmed from the “no prophet is accepted in his own village” syndrome. But after helping others develop and implement metrics programs, I now believe it’s deeper than that. Don’t get me wrong, the syndrome does make it difficult. Even if you are an outside consultant, the refrain will be raised—“Just ask the customer! Our customer satisfaction surveys are enough!” Not only will it be shouted, a chorus will rise up in strong harmony.

Even today, I have to fight for a multi-measure metric. Not just using more than two measures, but using ones from different views within the same quadrant.

I don’t mean to discourage you. In fact, I hope to do the opposite. I’m hoping that you use the concepts, tools, and techniques in this book to stop chasing data, stop using measures improperly, and to create meaningful metrics for improvement.

I hope that this book has provided you with what you need to plan and design effective metrics and I pray you have the strength of will to follow it.