CHAPTER 11

Using Earned-Value Management to Control Cost and Schedule Variance

The general who wins the battle makes many calculations in his temple before the battle is fought. The general who loses makes but few calculations beforehand.

—SUN TZU, THE ART OF WAR, 6TH CENTURY BC

Problems occur in projects for many reasons, including overly ambitious initial cost and schedule estimates, contracting problems, midstream changes to the project scope, problems installing hardware and systems software, staffing turnover, or coding and development complexity. These are but a few examples; there are myriad reasons why projects fall behind schedule and run over budget.

A primary source of project failures has been the lack of transparency; i.e., problems don’t become apparent to the project team and senior management as often as they should. When problems are initially encountered, project teams usually do everything within their power to address them. If the problems persist, however, the project team is often reluctant to report the problems and their impact to senior leadership.

The Department of Defense (DoD) has more experience than most agencies in dealing with the challenge of managing the development and acquisition of capital assets. DoD has a long track record of developing and acquiring large combat systems and other support infrastructure. Many years ago, DoD recognized that it needed a rigorous tracking and reporting methodology to hold contractors accountable for developing and delivering capital assets. In the 1960s it examined, adopted, and adapted the EVM approach for measuring and reporting cost and schedule performance. The EVM methodology is relatively straightforward, reasonably rigorous, and better than other methods available to projects and oversight groups.

What Is Earned-Value Management?

EVM is a methodology for making point-in-time comparisons of the estimated budget to the actual budget, and the estimated schedule to the actual schedule. Each point-in-time measurement calculates the variance between planned and actual costs and the planned and actual schedule. The variances are based on the approved budget and schedule, referred to as the baseline estimates, and are accurate and precise. Using EVM, a project manager can get an exact understanding of the relationship between funds spent and progress achieved toward the goal—to avoid, for example, a situation in which 80 percent of the funds have been spent but only 20 percent of the work has been completed.

EVM also uses the point-in-time variances to predict the final project cost and final completion date by assuming that the current levels of variance continue to the end of the project. These forecasts can be useful for their “shock value” when a project has a significant cost or schedule variance. Oversight groups receive the message that if some specific action is not taken to correct the cost and schedule variances, the forecasts will become the reality.

It is important to understand that EVM needs to be embraced by the project team and oversight groups and used vigorously by both groups to monitor progress, address issues as they arise, and ensure project success. Granted, it does necessitate additional rigor for project teams that may have used a more informal approach for estimating cost and schedule and tracking progress, but EVM’s systematic method of communicating progress is worth the effort.

EVM is best suited to tracking project performance. Projects differ from ongoing operations in that they have a beginning, middle, and end, necessitating a budget to explain how much it will cost to get to the end state and a schedule identifying when certain points in the project, including its completion, will likely be reached. EVM is useful for projects because they have estimates for the progress of their work, and events can still occur to disrupt achievement of those estimates within an acceptable timetable and within the estimated cost. EVM is less useful for ongoing operations, where there is an annual budget that is unlikely to vary, risk is low and controlled, and there is no anticipated end to the activities. In such cases, EVM is not useful because the cost and schedule variance is likely to always be zero. The only exception is if, within an ongoing operation, there are projects that are expected to begin and end within a finite timeframe. In those cases, EVM can be successfully applied to the project part of the budget.

The Relationship between EVM and CPIC

Details about how EVM works are provided in later sections of this chapter, but first the relationship of EVM to the other topics discussed in this book needs to be examined.

EVM is typically initiated when the advocates for an investment prepare a business case for review by an ITIRB. The business case must include budget and schedule forecasts that are based on a preliminary system design and associated estimate of all the tasks necessary to analyze, design, develop, test, implement, and operate the system. These tasks are captured within a visual framework that hierarchically displays the tasks, known as a work breakdown structure. As each task is identified, a work plan emerges with the associated costs and timetable to complete each work step. As the estimates are iteratively reviewed and finalized, the baseline project budget and schedule emerge and are incorporated into the business case.

It is important that the business case baseline not be submitted for ITIRB review and approval until it has been carefully reviewed and is as accurate as possible, because at the point of ITIRB approval the estimates constitute the equivalent of a contractual obligation. If, at the time the business case is reviewed by the ITIRB, the estimates are still not finalized, that fact must be clearly communicated to the ITIRB along with a date when the estimates will be final. Additionally, many agencies require that the baseline estimates be independently reviewed, given their importance in monitoring and controlling project team performance. (Chapter 12 discusses procedures for conducting an independent baseline review.)

The business case is an important element of the CPIC planning process because it affords the CPIC-SG and project team an opportunity to interact and discuss the project pathway to approval and to lay the groundwork necessary for the ITIRB review during the CPIC selection phase. The CPIC-SG can apprise the project sponsors and team regarding expected agency IT funding levels and any preparatory actions they should take to develop a persuasive argument, bring program leadership on board in terms of advocating the investment to the ITIRB, and ensure that the business case effectively communicates its expected return on investment (ROI). The project cost and schedule are key parts of the cost-benefit analysis that is required to be presented to the ITIRB.

EVM is most critical to the project once it is approved and development work commences. As part of the CPIC control phase, projects must routinely report EVM variance, usually on a quarterly basis. If variances exceed 5 percent of the baseline estimates, the project will be reported to the ITIRB; if they exceed 10 percent, the ITIRB will require that the project sponsor and manager meet with the ITIRB and present a mitigation plan to correct the variances. Interaction at this time is critical because the ITIRB must consider whether the project should continue based on the reasons for the variance and the likelihood that they can be corrected. If the ITIRB determines that the variances exist because the baseline estimates were not correct, it can require the project team to develop modified baseline estimates and submit them for approval. If this occurs, the ITIRB is likely to request an updated cost-benefit analysis and performance measurement targets to evaluate the financial impact of the modified baseline on the overall ROI.

The EVM performance information is also useful during the CPIC evaluation phase because, as part of a post-implementation review (PIR), the estimated and actual cost and schedule information, frequency and number of re-baseline approvals, and other project-related characteristics can be used as part of the evaluation criteria. Other performance information, including planned and actual outputs and outcomes, will also be used, but EVM provides useful insight regarding project team behaviors and performance.

Developing EVM Estimates

One of the most significant challenges in using EVM within the CPIC process involves the lead time necessary to prepare accurate cost and schedule estimates. The challenge exposes the tension that exists between obtaining project funding to pay for a project and doing sufficient work, at a fairly substantial cost, to be able to prepare accurate estimates. It is commonly accepted that sufficient analysis be done during the system development life cycle (SDLC) to fully identify requirements and expectations for a new IT system, before designing how to meet those requirements. This information collection enables participants to discuss and agree upon the scope of a project—what it will and will not do—before commencing the next steps to determine how to achieve the requirements. At this point in the SDLC, it is not possible to provide an accurate project cost and schedule baseline, and therefore the project is not ready to be presented to the ITIRB as part of the CPIC selection process.

After requirements and project scope are determined and agreed upon by sponsors and the project team, alternative approaches for achieving the objectives are considered. This phase involves analysis of both a status quo approach and viable alternative business and technical approaches, and it ultimately includes a quantitative cost-benefit analysis (as discussed in Chapter 9). Once an alternative approach is selected, a general design is completed, reviewed, and agreed upon. The general design then serves as the basis for the detailed design steps.

At the point where the general design is accepted and the detailed design is commencing, the project manager is able to prepare an initial work breakdown structure and the associated estimates of how long it will take to complete each task and how much it will cost. It will also be possible for the project manager to develop a sequencing plan for the work steps (sometimes referred to as the critical path), which is necessary to prepare a date-based project schedule.

A project must expend considerable resources just to reach a point where it can present an accurate set of baseline estimates for review by the ITIRB. Recognizing this reality and the importance of accurate cost and schedule estimates, project sponsors and managers should adjust their approach to seeking approval by organizing the investment initiative into a minimum of two phases. The first phase includes funding and a timetable for conducting an initial feasibility study, documenting project requirements and scope, conducting an alternatives analysis, preparing a general design, and developing the preliminary work breakdown structure and associated cost and schedule estimates necessary to submit an accurate business case to the ITIRB. The second project phase involves presenting the refined business case to the ITIRB and obtaining permission and funding to proceed.

Formal methods can be used to develop work breakdown structures and associated cost and schedule estimates. Approaches include successive levels of refinement, including developing a detailed control account plan (CAP) for each task included in the work breakdown structure. DoD requires rigorous application of such methods to control contractor performance for combat and other systems that typically cost millions of dollars. Smaller IT systems with fewer personnel typically do not need to refine their work breakdown structures to this level of detail, but project teams that have large numbers of personnel and are heavily dependent on contractor support might consider adopting advanced practices such as CAPs.

Project management tools, such as Microsoft^® Project Manager, typically include the capability to enter the project work breakdown structure and associated task details such as start and end dates, estimated cost, actual cost, and other EVM details. Often a project work plan will involve dozens or even hundreds of tasks and work steps. In such cases it is useful to roll up lower-level tasks into higher levels for reporting in the business case. Given that executives depend on the business case for communicating key milestones and deliverable dates, presentation of the work breakdown structure should represent critical achievement-points, but it does not need to include every work package. While there is no hard rule, the number of tasks and associated work packages in each business case should not exceed 25 to 30. On the other hand, presenting too few work packages and too little detail should also be avoided.

OMB encourages inclusion of key deliverables as part of the business case’s work breakdown structure because they provide an opportunity to assess tangible project elements and increase the reliability of the EVM reporting. Deliverables during early SDLC stages include feasibility studies, requirements documentation, and design documents. Development deliverables include deployment of system functionality, assuming that functionality and capabilities will be delivered in stages.

It is important that the project team accurately predict the project schedule and cost based on actual tasks (organized in the work breakdown structure as work packages). If the project manager expects that the project will get off to a slow start but will pick up the pace later, the baseline schedule should reflect that pattern. Some project teams tend to assume a normal distribution of work over time—that is, that 25 percent of the work will be done at the end of the first quarter, 50 percent at the end of the second quarter, and so forth. Such assumptions may not match reality. This is especially true when acquisition activities are involved because progress may be slow while contracts are being awarded, but rapid afterwards. Contractual issues can severely skew a project schedule and budget.

EVM Calculations

Once a project begins, it continues to spend money and consume time. Consequently, at any point after starting, the project manager can make his or her EVM calculations to determine if any cost or schedule variance has occurred. Although projects may be required to report EVM calculations to the CPIC-SG on a quarterly basis, monthly reporting is more useful, allowing the CPIC team to gauge progress and address any issues before they need to be reported to oversight groups.

To make EVM calculations, a project manager must have specific information on hand or must collect it. For each work package in the work breakdown structure, the project manager must know how many dollars have been spent up to the as-of date and what percentage of work has been completed for the work package. In the early stages of the project, only a few tasks will have such data. As the project progresses, more and more work packages will be started and will therefore have actual cost and percent complete data available. As mentioned earlier, it is useful to track EVM data using an automated project management tool; most of them include capabilities to capture and calculate EVM data.

Once the estimated cost and schedule and the actual cost and percent of work completed are known for each work package, a series of computations are done to determine cost and schedule variance and to use those variances to predict how much the final project will cost and when it will be done, assuming that the variances are not addressed. Table 11-1 lists definitions of EVM elements, and Table 11-2 presents the formula used to calculate the derived values. Generally, the project team and oversight personnel are more interested in the results of the calculations rather than the formulas.

Table 11-1    EVM Data Elements

Table 11-2    EVM Calculations and Formulas for Derived Fields

As noted, the important EVM elements include cost and schedule variance and the forecasted budget at completion and schedule at completion. Other data provided by EVM is also interesting and potentially useful. EVM produces two indexes, a cost-performance index and a schedule performance index, that correspond to the variances. The index scores are presented on a scale of 0.1 to 1.0 and represent project team scores or grades for how well they are managing the project budget and the project schedule. For example, a cost performance index score of 0.5 out of 1.0 is the equivalent of earning a grade of 50 out of 100 on a test, and a score of 0.8 out of 1.0 is the equivalent of earning a grade of 80 out of 100 on a test. Since most project managers are high-achievers and will respond to being scored, oversight personnel can use the performance indexes as a motivational tool.

EVM users should not be intimidated by the initial complexity represented by the fields and computations contained in Tables 11-1 and 11-2. The important elements are the cost and schedule variances, which ideally should have values of zero if planned and actual performances are perfectly equal. As variance increases, the need for investigation and action increases as well. When either variance reaches 5 percent, the project team should begin corrective action. When either or both reach 10 percent variance, the CPIC-SG should alert ITIRB members, request a formal analysis and report from the project team, and add the project to the schedule for the next ITIRB meeting. Action to mitigate significant variance, however, should not wait for an ITIRB meeting but should instead be taken immediately by the project team.

Other Considerations about EVM

Cost variance is generally a cause for concern. Once funds have been expended and exceed original estimates, it is difficult to recoup those costs later in the project. In short, once money is spent, it is difficult to find other places for cutting costs.

Some managers believe that the best approach for fixing cost overruns is to reduce project scope. From a CPIC perspective, however, reducing scope will impact the promised functionality as determined through performance measures as well as the cost-benefit analysis that was done to justify the project. There is no effective means of fixing cost overruns other than prevention.

Schedule slippages are also difficult to address. Some managers will initially think that the project can get back on schedule by adding more staff, but they will quickly recognize that doing so will increase cost variance. Specific actions that can be taken to address schedule variance include shifting tasks from sequential to parallel tracks, encouraging staff who are not compensated on an hourly basis (usually the case for government personnel) to work extra hours, or identifying other ways to gain scale economies or otherwise speed the process without compromising functionality or quality.

The American National Standards Institute (ANSI) has published an earned-value analysis standard (ANSI/EIA-748-A-1998) for using earned-value management to manage cost, schedule, and technical performance.¹ The standard contains 32 guidelines grouped into the following five categories:

   Organization

   Planning, scheduling, and budgeting

   Considerations

   Analysis and management reports

   Revisions and data maintenance

The standards represent desirable EVM qualities and operational considerations that provide for effective project management and control. Most agencies do not rigidly adhere to EIA-748, but do seek to understand and adopt the elements that best fit their needs and operating environment.

Agencies should ensure that contractors that support IT investments be required to use EVM techniques and methods for planning and conducting agency business. Inclusion of EVM requirements increases contractor accountability and gives the agency greater visibility and control over contractor efforts.

Reviewing and Challenging EVM Estimates and Reports

As noted earlier, ITIRB and CPIC-SG members should require that the EVM baseline be independently reviewed before it is finalized and submitted as part of the business case for approval and funding. The ITIRB and CPIC-SG are also responsible for questioning and challenging the EVM baseline estimates. This is best done by reviewing the business case as a whole and using the results of that review to ask both the obvious and the intuitive questions. The following issues are useful to explore as part of that discussion:

   Have the baseline cost and schedule estimates been risk-adjusted to account for those parts of the project that have significant risk? Provide examples of risks that were deemed significant enough to warrant adjustment to the cost and schedule and the magnitude of their impact.

   Have sufficient time and budget been allocated for intangible activities that occur as part of normal project dynamics, including meetings, project oversight, statement-of-work preparation, review of contractor proposals, development of transition software capabilities? Where are these costs reflected in the work breakdown structure?

   Does the work breakdown structure include associated operations and maintenance support that will be required once the investment is deployed? Does the period between initial deployment and mature operation include sufficient resources and time to ensure that all of the kinks are worked out of the system?

   Is there sufficient experienced staff to ensure that learning curves and turnover will not jeopardize project performance?

During the review process the work breakdown structure and associated budget and schedule should be reviewed to see if they have a linear distribution of project milestones and budget rather than a nonlinear distribution. A linear distribution sometimes reflects lack of detailed planning and development of a realistic critical path schedule. Project activities rarely flow smoothly and evenly over long periods. Activities such as competing and awarding contracts tend to cause projects to start more slowly and then to escalate and speed up later, leading to a nonlinear distribution of costs and milestones.

Project managers must also analyze variances. If they determine that a project has fallen behind schedule, they must consider whether tasks can be done concurrently rather than sequentially, take advantage of technical innovations to speed up the process, and determine if team members are being too exacting in adding small features and enhancements that are not necessary to meet requirements. Cost, as noted above, is a more intractable problem. Project managers may be able to reduce cost by using lower-cost facilities or products or deleting nonessential tasks.

The important message, whether dealing with cost or scheduling, is that all parties involved in project management and oversight must react proactively to cost and schedule variances. Some causal factors may be within the agency’s control and therefore can be addressed with collective approaches and involvement of senior agency officials.

Considerations in Modifying an EVM Baseline

In some cases it will be necessary to re-baseline a project. Re-baselining the cost and schedule estimates is necessary when original baseline estimates are no longer viable because either requirements or other circumstances were changed, or because the original estimates were invalid. When this happens, it is in the best interests of the investment team to first investigate the reasons initial estimates were invalid, and then to develop a revised baseline and to use it for monitoring cost and schedule variance.

Re-baselining has serious consequences. OMB perceives that a re-baseline request is indicative of a troubled project, and the project team must make a considerable effort and convincing arguments for OMB to agree to a funding increase. Unless a legitimate reason unrelated to the project exists, OMB is likely to instruct the agency to take the needed funding from other agency programs. In short, re-baselining cost estimates means that funding will have to be shifted from some other program to cover the increase in costs.

Clearly, project estimates must be accurate before a final baseline is approved. And the project team must then carefully manage its work according to the approved cost and schedule. Anything less will create considerable additional work and could also damage agency and personal reputations along the way.

With that said, initial estimates, if submitted before sufficient work has been done to ensure their accuracy, should be identified as preliminary rather than final estimates, and a date when the final estimates will be available should be provided. Once final estimates have been made, an independent baseline review should be conducted to ensure that the estimates are as accurate as possible and that all key factors have been taken into consideration.

EVM as a Key CPIC Tool

EVM is a useful project management and reporting methodology that supports all CPIC phases. EVM is an essential tool for selecting investments and one of the primary mechanisms for monitoring and controlling progress and project team performance. EVM is now a mandatory element of government IT procurements because it increases contractor accountability. For the same reason, it is also mandatory for IT development teams.

ENDNOTES

1. National Defense Industrial Association (NDIA) Program Management Systems Committee (PMSC), “Earned Value Management Systems Intent Guide,” November 2006 Edition. Online at http://management.energy.gov/documents/NDIA_PMSC_EVMS_Intent-Guide_Nov_2006.pdf (accessed December 2007). This Intent Guide was created by NDIA PMSC to provide additional insight into the Earned Value Management System (EVMS) guidelines included in Section 2 of the ANSI/EIA-748-A Standard for Earned Value Management Systems. For more information, visit the ANSI website at http://www.ansi.org (accessed December 2007).