Using Costs-Consequences Analysis in Any Organization
Using the OEM and the concept of costs-consequences, the following questions allow for the identification of what was missing for useful evaluation and continuous improvement to occur:
• What data exist?
• What data do not exist?
• What are critical data required by the project/intervention to determine effectiveness, efficiency, and positive costs-consequences?
• What Organizational Elements data exist?
• What Organizational Elements data are missing complete and valid data?
• Are the gaps between What Is and What Should Be identified?
• Are all of the Organizational Elements linked? (Does a “flow” from element to element exist and is it justified?)
Why do this? It makes sense because when you are required to report on return on investment, or costs-consequences, this provides some guidelines for demonstrating “what you give” and “what you get” for actual or potential interventions. You can provide hard metrics for value added (or value subtracted if things don’t go as intended).
The Bernardez Two-Level Business Case Model. Recall that in Chapter 3, Table 3.2, Bernardez’s two-level business case (conventional bottom line and societal bottom line) provides an approach to proving value for money. This two bottom lines business case can be very powerful in showing return on investment for both conventional calculations and for societal ones as well. Demonstrating intentions and success can thus be based on demonstrating the cost to meet the needs as compared to the costs for not meeting the needs.
Interestingly, much of what practitioners are advised to do that get called “needs assessments,” “needs analyses,” “training needs assessments,17” and the like are usually quasi-needs assessments because they deal with gaps in processes (e.g., training, job aids, methods of supervision, resources desired) and not with results.
Some practical considerations. Needs assessment, as defined here, is a planning process for (a) identifying the gaps between current results and required/desired ones, and (b) placing those needs (gaps in results) in priority order on the basis of the costs to meet the needs as compared to the costs to ignore them. Since a need is defined as a gap in results, then there are three types of needs: one each for gaps for Mega level results, Macro level results, and Micro level results.18 Gaps in non-results, Processes and/or Inputs, are termed quasi-needs. You can do needs assessment at the Mega, Macro, or Micro level by identifying gaps in results for each.
By identifying correct and important needs before implementing any process or solution, you can improve your effectiveness and efficiency. Securing the necessary information for “selecting the right job” in order that doing the job correctly will be fruitful can do this. Often, when dealing with an ongoing system, evaluation data can supply you with the What Is data, but it cannot give you the What Should Be.”
Action Steps
1. Needs are gaps in results, not gaps in means or resources. Treat “need” as a noun, not a verb.
2. Before sensibly selecting means and resources, you should know the needs—gaps in results—to be closed. Don’t jump to solutions before you know the problems.
3. A useful needs assessment identifies gaps in results at each of three levels of planning and results, and places those needs in priority order on the basis of the cost to meet the need versus the costs to ignore the need. Doing it this way gives you a triple bonus:
• The What Is part of the assessed needs serves as your objective.
• The gaps between What Is and What Should Be will provide the basis for evaluation by revealing how much of the need has been reduced or delivered.
• By using “need” as a gap in results, you can demonstrate the costs of meeting the needs as compared to the costs to ignore the need. This is very powerful for proposals.
4. What commonly gets called a “needs assessment” is usually about gaps in means and resources and might more accurately be called a “solutions assessment.”
5. Get commitment of all partners to this approach to needs assessment.
6. A needs assessment should use both perception data (soft data) and performance data (hard data), and you should ensure that both sources of data agree.
7. Don’t use a “training needs assessment” even as popular as they are. Research has shown that if you base activities on the basis of that you will be wrong 80 to 90 percent of the time.
8. Stay consistent and rigorous. Use the Six Critical Success Factors constantly.
Endnotes
1. Of course, a dictionary will tell you “need” can be used as a verb. Be warned, however, when you do use it as a verb or in a verb sense you risk jumping into solutions before knowing the real problems, confusing means and ends. The interesting thing about dictionaries is that they give “common usage,” not necessarily precise or useful usage. Semantics and words are important, so please give consideration to the specific words and concepts used in this book; the precise use of words are important, even when at first it all seems like semantic quibbling. It is not.
2. Kaufman, R. (1976). Needs assessment. San Diego, CA: University Consortium for Instructional Development and Technology.
Kaufman, R., & English, F. W. (1979). Needs assessment: Concept and Application. Englewood Cliffs, NJ: Educational Technology Publications.
Kaufman, R. (1986a). Obtaining functional results: Relating needs assessment, needs analysis, and objectives. Educational Technology, 26(1), pp. 24–27.
Kaufman, R. (1992). Strategic planning plus: An organizational guide (revised). Newbury Park, CA: Sage Publishing.
Kaufman, R. (1995). Mapping educational success (revised). Thousand Oaks, CA: Corwin Press.
Kaufman, R. (1998). Strategic thinking: A guide to identifying and solving problems (revised). Washington, DC and Arlington, VA: The International Society for Performance Improvement and the American Society for Training and Development.
Kaufman, R. (2000). Mega planning: Practical tools for organizational success. Thousand Oaks, CA: Sage Publications. Also Planificación Mega: Herramientas practicas paral el exito organizacional. (2004). Traducción de Sonia Agut. Universitat Jaume I, Castelló de la Plana, Espana.
Kaufman, R. (2006a). Change, choices, and consequences: A guide to Mega Thinking and Planning. Amherst, MA: HRD Press Inc.
Kaufman, R. (2006b). Seven stupid things people do when they attempt strategic thinking and planning. In Silberman, M., & Phillips, P. The 2006 ASTD organization development and leadership sourcebook. Alexandria, VA: ASTD
3. What about Maslow and his 1964 widely revered “Hierarchy of Needs”? Actually, using terms as defined here, Maslow’s Hierarchy is actually a hierarchy of “motivators”; it identifies the rough order in which an individual will be motivated to close gaps, from survival to self-actualization. Useful, but not really “needs,” but rather motivators.
4. Each of these was mined from published so-called needs assessments. Based on previously published work, including:
Kaufman, R. (2000). Mega Planning: Practical tools for organizational success. Thousand Oaks, CA: Sage Publications. Also Planificación Mega: Herramientas practicas paral el exito organizacional. (2004). Traducción de Sonia Agut. Universitat Jaume I, Castelló de la Plana, Espana.
Kaufman, R. (2006). Change, choices, and consequences: A guide to Mega Thinking and Planning. Amherst, MA: HRD Press, Inc.
5. Based in part on Kaufman, R. (2000). Mega Planning: Practical tools for organizational success. Thousand Oaks, CA: Sage Publications.
6. Based on Kaufman, R. (1992). Strategic planning plus: An organizational guide (revised). Newbury Park, CA: Sage Publishing. and Kaufman, R., Rojas, A. M. & Mayer, H. (1993). Needs assessment: A user's guide. Englewood Cliffs, NJ: Educational Technology.
7. Deming, W. E. (1982). Quality, productivity, and competitive position. Cambridge, MA: MIT, Center for Advanced Engineering Study.
Deming, W. E. (1986). Out of the crisis. Cambridge, MA: MIT, Center for Advanced Engineering Technology.
Deming, W. E. (1990: May 10). A system of profound knowledge. Washington, DC: Personal memo.
8. Juran, J. M. (1988). Juran on planning for quality. New York: The Free Press.
9. Kaufman, R., Thiagarajan, S., & MacGillis, P. (Eds.). (1997). The guidebook for performance improvement. San Francisco, CA: Pfeiffer & Co./Division of Jossey-Bass.
Muir, M., Watkins, R., Kaufman, R., & Leigh, D. (April, 1998). Costs-consequences analysis: A primer. Performance Improvement, 37(4), pp. 8–17.
10. Special journal issues on Mega:
Kaufman, R., & Bernardez, M. (Eds.) (2005) Performance Improvement Quarterly, 18(3).
Kaufman, R., Bernardez, M., & Guerra-Lopez, I. (Eds.) (2009). Performance Improvement Quarterly. 22(2).
11. These are based on Chapter 4 of Kaufman, R., & Guerra-Lopez, I. (2008), The Assessment Book: Applied Strategic Thinking and Performance Improvement through Self-Assessments. Amherst, MA: HRD Press, Inc., and are truncated here.
12. From other previously cited works by Kaufman.
13. Based on Kaufman, R. (1992). Strategic planning plus: An organizational guide (revised). Newbury Park, CA: Sage Publishing, and Kaufman, R., Rojas, A. M., & Mayer, H. (1993). Needs assessment: A user’s guide. Englewood Cliffs, NJ: Educational Technology.
14. Kaufman, R. (2006). Change, choices, and consequences: A guide to Mega Thinking and Planning. Amherst, MA: HRD Press, Inc.
15. This section is based on:
Kaufman, R. (1998). Strategic thinking: A guide to identifying and solving problems. (revised). Washington, D.C., and Arlington, VA: The International Society for Performance Improvement and the American Society for Training and Development.
Kaufman, R. (2000). Mega Planning: Practical tools for organizational success. Thousand Oaks, CA: Sage Publications.
Oakley-Browne, H., Watkins, R., & Leigh, D. (2003). Practical strategic planning: aligning people, performance, and payoffs. San Francisco, CA: Jossey-Bass/Pfeiffer.
Kaufman, R. (2006). Change, choices, and consequences: A guide to Mega Thinking and Planning. Amherst, MA: HRD Press, Inc.
16. Interestingly, most “return on investment” models and procedures leave out Mega. They might reach to customer satisfaction and “contributions,” but do not deal with adding measurable value to both external clients and our shared society. In work previously cited by Bernardez, an alternative business case model is now available.
17. Please notice that in my list of important definitions, “training needs assessment” wasn’t there. I believe that this is a misnomer for a “training requirements analysis,” which is very important after you know that training (or some intervention is required). After all, training is a means—why would you want to do a needs assessment (determining gaps between current results and desired ones) if you already know that you are going to do training? Again, even the label of one of our favorite tools supplies a bias toward means (training) while assuming that useful ends will surely follow.
18. These three types of results, unfortunately, are not often distinguished in our literature…you would think that “if you’ve seen one kind of result, you’ve seen ‘em all.”