CHAPTER 9
Alternatives Analysis
If you really want to do something, you’ll find a way.
If you don’t, you’ll find an excuse.
—JIM ROHN, BUSINESS PHILOSOPHER
For the CPIC process to have integrity, both the ITIRB and the CPIC support group (CPIC-SG) must set high expectations for investment teams to present a well-thought-out set of alternative approaches for achieving the business goals. The results of such an alternatives analysis are critical to investment decision-making because well-chosen alternatives drive innovation and efficiency. If effective investment management depends on selecting investments based on price, performance, and risk, the alternatives analysis and associated cost-benefit analysis are the source for two out of the three decision factors.
In reviewing the alternatives, the ITIRB must ensure that:
Project teams did not simply justify the alternative that they preferred rather than examine a range of alternatives, then “cook the numbers” to prove what they wanted to prove
A broad range of alternative business and technical approaches for accomplishing the investment objectives have been identified and evaluated
The cost-benefit analysis is rigorous, the useful life estimated for the analysis is realistic, cost and benefit forecasts take into account the time-value of money, and sensitivity analysis is done to test assumptions that were made
A model of the alternatives and associated CBA has been developed that can be updated over time as estimated costs of the selected alternative become actual costs
Each of these issues will be addressed in more detail, but the important initial point is that the ITIRB should not simply accept a project team’s information without challenging its underlying assumptions and ensuring that an appropriate and thorough methodology was used. The ITIRB can do so by requiring that an independent alternatives and associated CBA be conducted. An independent group with sufficient technical skills can work with the project team to challenge and review inputs, assumptions, calculations, and results; to improve the quality of the analytical results; and to render a second opinion regarding the final product. Independent parties might come from the agency’s inspector general’s office, program analysis office, or the CFO’s office.
A summary of the alternatives/CBA results is presented to the ITIRB as part of the investment business case. The information should describe how the alternatives were identified, which ones were easily eliminated from additional analysis and which ones were included, and which alternative, based on an analysis of intangible benefits and the cost-benefit analysis, was selected.
Identifying and Selecting Alternatives
One of the most daunting challenges confronting some project teams is understanding what is meant by “alternatives” in the context of developing an IT system. This confusion has been fueled by OMB and media discussions that appear to contradict each other regarding the nature and number of alternatives that should be considered, and how to treat status quo systems in place.
The ITIRB should expect that the investment team has exercised due diligence in identifying innovative solutions rather than just rationalizing its preferred approach. It is inappropriate for an investment team to select the approach that it prefers and to then arbitrarily identify and analyze two obviously inferior options to get through the CPIC process. Therefore the business case should describe what research was done to identify and investigate best practices and viable alternatives. Source material might include scholarly articles, trade publications, research reports, white papers, and other formal studies.
Table 9-1 illustrates various ways to construct alternatives. The variety of alternatives illustrates that technology adoption is an inherently creative process that requires some degree of out-of-the-box thinking. Some of the most successful innovations result from scanning and considering how current capabilities can be used to achieve extraordinary results. Project teams should be encouraged to vigorously engage in the alternatives analysis process.
Often, there is a tendency to treat the current level of automation as a distinct alternative—that one possible decision is to do nothing and to continue on the present path. It is important that the status quo not be included as an alternative because the cost-benefit analysis must be structured in terms of how costs and benefits will change relative to the current situation. Including the status quo would result in zero additional cost and zero additional benefit, necessitating a “divide-by-zero” formulation. Knowing what the costs and benefits of the current technology system are is useful information, but the current system should not be introduced as a viable alternative.
Table 9-1 Approaches for Constructing Alternatives
Alternative |
Description |
Cost-Benefit Considerations |
1. In-House or Contractors |
Alternative approaches for developing the system. Might include developing the system using in-house resources, using contractors, or using a combination |
Alternatives have different cost structures because contractors generally tend to be more expensive than in-house personnel, but they are unlikely to produce higher benefits because the same system will be implemented regardless of who develops it. |
2. Build or Buy |
Alternative approaches for developing in-house, purchasing an off-the-shelf package, or using an available e-government application. Alternatives might include build, buy without customization, or buy with customization. |
Alternatives have different cost structures and different implementation timetables because it takes longer to build a system than to acquire one. Hence the pre-built system is likely to realize benefits more quickly. |
Alternatives might be structured around location of the system. For example, it might reside on a mainframe or on a headquarters-based file server, or it might be distributed throughout the organization. Another option is to house the system on a service provider’s location. |
Alternatives likely will have different cost structures, but the benefits may not vary across the identified alternatives assuming that the system will provide the same level of functionality regardless of where it resides. |
|
4. Technology Platform |
Alternatives might include deployment on a mainframe, on server-based/web- enabled applications, on a web server, or as part of web services. |
There is a range of technical options that can be considered for designing and developing the system. Costs and benefits might vary depending on the alternative. |
5. Technical Approach |
Alternative technical approaches offer significant opportunity for innovation, including adopting the latest technology such as wireless, handheld technology; radio-frequency identification; or other capabilities to accomplish the business requirements. |
In most cases, each of these alternatives will have different costs and benefits. Costs and benefits may increase as more advanced alternatives are examined—the key reason why the cost-benefit ratio is an important element of the analysis. |
6. Business Approach |
Alternatives include different ways to approach meeting requirements from a business perspective, such as centralizing data at the federal level or within a single agency versus decentralizing data closer to the point of data capture. Many other structural and relationship factors can be considered in some cases for best using technology to achieve the desired outcomes. |
In most cases, each of these alternatives will have different costs and benefits. Costs and benefits may increase as more advanced alternatives are examined. Like the preceding category, this is a key reason why the cost-benefit ratio is an important element of the analysis. |
The Federal Approach to IT Cost-Benefit Analysis
OMB has classified IT systems as capital assets and instructed federal agencies to treat them as such. Consequently, agencies are required to identify alternative approaches and develop a cost-benefit analysis for each alternative. Results can then be compared, and an informed choice can be made from among the alternatives.
To emphasize the benefits of an investment, OMB refers to the analysis as a benefit-cost analysis (BCA), defining it as “the present value of benefits divided by the present value of costs, minus one, expressed as a percent.”1
OMB provides specific instructions for conducting a BCA in its Circular A-94, where it defines BCA as “a systematic quantitative method of assessing the desirability of government projects or policies when it is important to take a long view of future effects and a broad view of possible side effects.”2
It is beyond the scope of this book to provide a thorough guide to conducting a BCA. However, there are certain challenges, including some that are unique to analysis of information systems, that merit discussion. Addressing them helps inform CPIC decision makers and offers insight to project teams conducting a BCA that might not be available from other sources. Certain BCA elements must be present to ensure a thorough analysis and to meet OMB Circular A-94 requirements. They will be highlighted in the report to ensure that sufficient work has been done when considering alternatives and preparing the BCA.
In performing a BCA, it is important to determine the useful life of the proposed investment. This is relevant because both benefits and costs will have to be estimated for the entire useful life. It may be tempting to rationalize that technology changes so much that the useful life should be only four or five years, but doing so creates a disadvantage for a BCA computation because it reduces the number of years that benefits will be realized and therefore makes it more difficult to justify the costs to build or acquire the investment. In general, the useful life of an IT investment ranges from eight to ten years. Some systems, especially in the federal sector, remain active for decades.
Investment Benefits
The business case must include two categories of benefits: intangible ways that the investment benefits agency activities, outputs, and outcomes; and tangible benefits. The former category will not be factored into the quantitative analysis and therefore must be included in a preliminary section of the alternatives analysis. Anyone who has worked in government for any length of time recognizes that IT systems sometimes improve operations in ways that cannot be measured in dollars. Such benefits must be included in the business case and measured using non-dollar methods. For example, the time that it takes to get, say, a response such as a license, letter, or document to a citizen might be significantly reduced. There may also be cost savings that can be quantified, but improving public satisfaction generally cannot be translated into dollar values.
Table 9-2 presents examples of how intangible activity, output, and outcome improvements might be communicated in the business case; the benefits are actual examples taken from OMB’s federal enterprise architecture.
Table 9-2 Measuring Intangible Benefits
It is also important for an investment to demonstrate some level of financial benefit. OMB is more likely to support funding for an investment that has a positive return. For inexperienced analysts, engaging in the process of identifying tangible financial benefits may initially be a bit daunting. After carefully considering what will happen when the investment is deployed, however, various types of quantitative benefits will emerge.
The most obvious benefit offered by many investments is saving time. A government employee who normally took eight hours to do a task but now only takes three hours is saving five hours per person per day. To make a final quantification, these hours must be translated into dollar terms. Procedures for doing so are available in OMB Circular A-94. Another example of a quantifiable benefit is when a new investment diverts people from tasks that have no added value to value-added tasks. Counting those hours produces tangible benefits because diverted hours are the equivalent of hiring new persons to do the value-added tasks. Analysts can calculate this benefit by adding the number of hours converted from non-value-added to value-added activities and using procedures in OMB Circular A-94 to convert them into salary savings.
Other tangible benefits involve reductions and savings in non-personnel costs, such as reductions in office space, travel costs, postage, contractor costs, or other similar categories. Estimation of these benefits is generally a straightforward exercise.
One consideration that should be taken into account is whether the included benefits will result in an opportunity to actually lower the budget for those entities that will realize the benefits. Analyses should include a statement indicating if the benefits create capacity to handle increased workload or more complex tasks, or whether fewer personnel or contractors or less space will be needed once the investment is implemented.
Investment Costs
It is generally easier to estimate investment costs than to estimate benefits. The costs are equal to the estimated budget for the entire life of the asset. Cost estimates must take into account all costs during the system development life cycle (SDLC), as well as the subsequent operations and maintenance costs necessary to support the investment throughout its useful life.
Varying approaches are employed across federal agencies regarding the question of whether the cost of government personnel must be included in the project budget, since the personnel will be paid regardless of a decision to proceed with a particular system development project. From a BCA perspective, however, actual costs must be used, and that includes the cost of government personnel. Actual costs also include the cost of users who participate in design and implementation activities and transitional activities such as training users prior to implementation.
The rule of thumb for which costs to include in the BCA is as follows: any cost that will be incurred if the system is built should be included in the cost estimate. Do not include costs that will be incurred regardless of the outcome of the business case decision. For example, an agency whose work included collecting data, using statistics to analyze the data, and publishing statistical reports was deciding which costs to include. The agency had a large staff dedicated to analyzing collected data and developing statistics. The business case authors were uncertain if the costs of that staff should be included in the BCA. The answer is no because those personnel would do their jobs regardless of whether a new automated system was implemented to collect the data or if the data were collected manually. Their costs should not be included as part of the investment BCA analysis.
Analysts should include all costs for planning, design, development/acquisition, customization, operations, and maintenance. The following cost categories are representative of the types that should be included:
Development and program employee resources (labor hours)
Contract services
Software licenses and off-the-shelf software
Hardware and equipment (e.g., cabling, power supplies)
Additional facilities needed for the investment
Transition costs
Training and administrative support necessary to support the investment
Other (e.g., consumable supplies required for the investment)
As noted earlier, the costs for each of these categories must be shown on an annual basis for the entire useful life of the project. This is necessary because the costs must be adjusted to account for the time value of money. The calculation is usually done using software, rather than manually.
Any assumptions about cost estimates should be fully explained, and all cost estimates must be adjusted based on identified project risk. For example, if a new technology is being used, additional costs should be added during the planning phase to provide training to the project team.
Other Cost-Benefit Considerations
Once viable alternatives have been selected, conducting the BCA is a reasonably straightforward process. As already noted, investment analysts must identify and quantify costs and benefits for the useful life of the investment. This means that on a year-by-year basis for an investment’s life cycle the funds that will be spent and the dollar benefits that will be realized need to be documented.
This format is important because financial analysis that involves comparing dollars over long time periods necessitates taking into account the time value of money, which is based on the fact that a dollar at any given time is likely to be worth less or more than a dollar at some other time. OMB Circular A-94 therefore provides discounting procedures that must be used to address the time value of money.
OMB’s criterion for comparing investments is based on economic principles and is referred to as net present value—the discounted monetized value of expected net benefits (i.e., benefits minus costs). According to OMB:
Net present value is computed by assigning monetary values to benefits and costs, discounting future benefits and costs using an appropriate discount rate, and subtracting the sum total of discounted costs from the sum total of discounted benefits. Discounting benefits and costs transforms gains and losses occurring in different time periods to a common unit of measurement. Programs with positive net present value increase social resources and are generally preferred. Programs with negative net present value should generally be avoided.3
OMB also addresses issues of uncertainty. Investment benefit and cost estimates are typically uncertain because the underlying data are imprecise and certain assumptions have to be made in developing the cost and benefit estimates. Consequently, the effects of uncertainty and associated assumptions must be analyzed and reported. Factors might include sources of uncertainty and how sensitive the results of the BCA are to those sources of uncertainty.
A useful technique for evaluating the impact of certain assumptions of the BCA is sensitivity analysis. Projects rarely proceed under optimum conditions, yet most IT justifications are based on an ideal scenario that does not take into account the likelihood that something will go wrong. Sensitivity analysis addresses this problem.
Most software tools that provide BCA capabilities include features for conducting a sensitivity analysis. The process involves changing the estimates of key assumptions, varying them so that the estimates are much higher and much lower, and examining the impact of these variations on the project’s return on investment (ROI) and benefit-cost ratio. At some point, as an assumption is varied significantly, one of the other alternatives may become the preferred alternative. By noting how much variation it took to change the decision, the analyst will be able to report how far the estimates could be off before the selected alternative is no longer the best one.
Sensitivity analysis can be conducted on all estimates that might ultimately have some variation during the project. Contractor costs might vary, especially if a contract has not yet been awarded. Software and hardware cost estimates might vary depending on how much background research was done. Sensitivity analysis should also be a required part of a BCA. If members of the ITIRB do not understand sensitivity analysis, they should request presentations on it from experts. The ITIRB should ensure that sensitivity analysis is done and be prepared to ask questions about the sensitivity of various assumptions.
Presenting Alternatives Analysis Results
The results of the alternatives analysis should be described concisely but thoroughly, emphasizing the rationale that was used to make the selection. In some cases, the selected alternative may not have the best benefit/cost ratio (BCR) or ROI. In such cases, a strong justification for the selection should be made, including inclusion of both qualitative and quantitative criteria. The comparison of alternatives should address the ROI, as required by OMB.
Periodic Updates to the Alternatives Analysis
The ITIRB should require that each investment update the alternatives/BCA each year as part of the planning and selection phases. This is not just a waste of time. It is a rational exercise for incorporating actual instead of estimated costs to determine if the ROI has changed and if the initial investment decision is still prudent. Variances from the anticipated return alert management that the asset is not being implemented or performing as intended and that action is needed.
ENDNOTES
1. Office of Management and Budget, Circular A-94 (Revised): Guidelines and Discount Rates for Benefit-Cost Analysis of Federal Programs, October 29, 1992. Online at http://www.whitehouse.gov/omb/circulars/a094/a094.html (accessed December 2007).
2. Ibid.
3. Ibid.