Evaluation: Different Yet Related to Needs Assessment
Needs assessment and evaluation are related but not the same. Let’s take a look at how they are related, how they are different, and how both are key to successful planning and doing: Planning is proactive; it seeks to define and achieve a useful future.
• Evaluation is reactive; it finds what worked and what did not.
• Evaluation compares results with intentions.
• Evaluation is after-the-fact. Needs assessment and planning is before-the-fact.
• Evaluation data should be used ONLY for fixing and improving, NEVER for blaming.
• The basics of evaluation are not complex, but you might think so from the literature and all of the “cult followings” and rituals.
• Evaluation data—comparing “what was accomplished” with “what was intended” is best used for continuous improvement. Continual improvement is the process for changing what should be changed whenever you realize that a change is required.
• Evaluation should use two kinds of data: “hard” and “soft.” Hard data are independently verifiable, and soft data are personal and based on perception. Make sure that both kinds of data agree.
• Needs assessments should also include both hard and soft data, but these data only relate to finding the gaps between current results and desired ones.
• When Mega is considered in evaluation, it is called “Evaluation Plus” to indicate that it goes beyond conventional evaluation frameworks.
Needs assessment and evaluation: different but related. Both needs assessment and evaluation deal with gaps, but for different purposes. Needs assessments and evaluation are related but different.
When you do needs assessment, you are, before the fact, finding the gaps between current results and desired ones: a comparison of What Is with What Should Be or “could be.” Evaluation is reactive, needs assessment is proactive.
When you evaluate you compare, after-the-fact, the results that were obtained with the results you intended to accomplish: a comparison of what was accomplished with what was intended.
Needs assessment and evaluation pose different yet related questions. Both compare intended with actual results, but evaluation is always after-the-fact—comparing results with intentions—while a needs assessment proactively identifies the gaps between current results and desired results.
Needs assessment data provide the criteria for evaluation, and evaluation provides the data about what was actually accomplished and delivered.
Because the two are related (and often confused in conventional usage), let’s take a closer look at the relative of needs assessment: evaluation.
Figure 9.1 provides a tool for guiding you in evaluation and continual improvement.
Figure 9.1. A job aid for deciding on what to do for evaluation and continual improvement.
Proactive and reactive tools: Needs assessment and evaluation. Needs assessment is a proactive planning tool, and is distinct from evaluation. Evaluation is a retrospective, or after-the-fact, concern with gaps between what was achieved through our efforts and the targets we have previously set for ourselves.
Needs assessments are forward looking and determine gaps between our current results and those we should obtain. Needs assessments are not limited to existing objectives, but may be used in deriving new ones.
• Evaluation compares results with intentions.
• Evaluation is after-the-fact. Needs assessment and planning is before-the-fact.
• Evaluation data should be used only for fixing and improving, never for blaming.
• The basics of evaluation are not complex, but you might think so from the literature and all of the “cult followings” and rituals.
Evaluation of Mega Planning
Applications: Some Cases in Point
There are a number of applications of Mega Thinking and Planning in most parts of our world ranging from Australia, New Zealand, Canada, Central America, South America, to Europe and Asia. Practical applications include the Australian Department of Defense, New Zealand Army, U.S. Coast Guard, and the government of Panama, among others.
Specific uses include African Women’s health, public employee union and management collective bargaining, vocational rehabilitation, and social change.1
The following three specific cases provide examples and performance data from which we can learn and that demonstrate the viability of the approach.
The Refinor Case
Argentina was home to a major gas production and distribution company. YPF, that was privatized and purchased by Refinor. Mariona Bernardez became the key management consultant who applied Mega Thinking and Planning to that operation to ensure it was a viable business as well as dealing with its social and human resources obligations. With successful implementation, Refinor was sold to Petrobras who went back to conventional bottom line thinking and management. Bernardez traced key indicators for (a) hours of social work invested in community projects, (b) market share of the regional fuel market, (c) local unemployment rate, (d) riots, incidents, and disruptions reported by the police, and (e) property loss.
Data in Figure 9.2 show that indicators during the application of Mega improved during that time period, but was wanting before and after Mega.
| Strategic Performance Indicators (SPI)
| |
| Indicator (SPI) | Definition |
|
| |
| Hours of social work | Hour of social work included the time invested in community projects by the company and volunteers |
| Market share | Refinor’s share of the regional fuel market |
| Local unemployment rate | As defined by statistics from INDEC and government of Salta for Aguaray, Pocitos, Tartagal, and Mosconi, the main communities surrounding Refinor’s refinery and main infrastructure. |
| Riots and incidents | As reported by local police |
| Property loss | As reported by local authorities |
SPI Evolution Chart
Figure 9.2. Data relative to an organization shifting to Mega Thinking and Planning and then back again. The societal contributions and impact are severally addressed.
This is a before, during, and after assessment of the application of Mega Thinking and Planning.
The Sonora (Mexico)
Institute of Technology
Almost two decades ago, the management and faculty of a technical university adopted societal value added and wealth creation for the entire organization. They went through several phases, Planning, Implementation, Evaluation of Curricular Results, Evaluation of Social Impact, and Evaluation of innovation and leadership. Now six campuses, they adopted an Ideal Vision and mission and also created a Ph.D. and an MBA in performance improvement. As they progressed, they also created more than a dozen incubator projects (such as a Software Factory) that, using Mega, developed viable businesses. The university has been recognized as one of the innovative institutions of higher learning in Mexico and recognized by the president of Mexico and now has the choice to continue to be innovate and practical.
Proposed Transformation of a Major Central American City2
The second largest city in Panama has struggled while the capital had developed and prospered. A team was invited by the executive of this country to demonstrate how Mega Thinking and Planning could transform the city that was in crisis. The proposed business case and rationale follow in Figure 9.3.
Figure 9.3. Applying Mega Thinking and Planning to the transformation of a city and applying the Bernardez 2-level business case, projects that a positive return-on-investment—if the initiative is management completely—could start in the first year.
This application integrates the many aspects of a city and community and considers them as a whole—as a system—in identifying the costs and consequences of all variables, not just one or a few. Such Mega Thinking and Planning can offer options not usually recognized in programs of planned change and transformation.
These examples provide justification of why Mega Thinking and Planning is valid and useful. The example here shows the power of this holistic approach that Mega Thinking and Planning takes. It is “a system approach” because it considers all of the variables on the entire system instead of just focusing on one or two subsystems. Because all of the subsystems do interact, this takes that dynamic relationship into account. It makes the likelihood of delivering success and proving it a reality.
Mega Thinking and Planning works. It may be successfully applied in a wide range of areas in the public as well as private sectors. It can define and deliver success and add value— measurable value—to all stakeholders.
Action Steps
1. Realize that Mega Thinking and Planning as well as needs assessment are proactive. They are used to create a future based on hard evidence.
2. Realize the evaluation, as important as it is, is reactive. You can only evaluate what has happened.
3. Both needs assessment and evaluation use the same data: the gaps between current results and desired results.
4. Use evaluation data only for fixing and never for blaming.
5. Evaluation is based on data—data from the gaps between What Is and What Should Be.
6. Mega Thinking and Planning has been successful in a wide range of applications, cultures, and organizations, so use with confidence.
Endnotes
1. Kaufman, R., & Bernardez, M. (Eds.) (2005). Special issue on using societal value added as the practical alternative to conventional failure. Performance Improvement Quarterly, 18(3), and Kaufman, R., Bernardez, M., & Guerra-Lopez, I. (2009). (Eds.). Special invited issue on Mega planning. Performance Improvement Quarterly, 22(2).
2. Developed by Mariano Bernardez in response to a cabinet level request to a team comprised of Carmen Arias, Adam Krivatsy, Mariano Bernardez, and Roger Kaufman for a business case for the application of Mega planning to transform the entire city of Colon.