The purpose of Organizational Process Performance (OPP) is to establish and maintain a quantitative understanding of the performance of the organization’s set of standard processes in support of achieving quality and process-performance objectives, and to provide process-performance data, baselines, and models to quantitatively manage the organization’s projects.
Process performance is a measure of actual results achieved by following a process. Process performance is characterized by process measures (e.g., effort, cycle time, and defect removal effectiveness) and product measures (e.g., reliability, defect density, capacity, response time, and cost).
The common measures for the organization consist of process and product measures that can be used to characterize the actual performance of processes in the organization’s individual projects. By analyzing the resulting measurements, a distribution or range of results can be established that characterize the expected performance of the process when used on any individual project.
Tip
A subprocess measure’s “central tendency and spread,” normalized appropriately (e.g., for work product size), can serve as its process performance baseline (PPB). Such a PPB can be displayed in different ways, such as by a control chart, box plot, or histogram. OPP practices assume, at a minimum, that PPBs are established for selected subprocess measures, but the organization and projects may benefit from establishing PPBs for other measures too.
In this process area, the phrase quality and process-performance objectives covers objectives and requirements for product quality, service quality, and process performance. As indicated earlier, the term process performance includes quality; however, to emphasize the importance of quality, the phrase quality and process-performance objectives is used rather than just process-performance objectives.
Measuring quality and process performance may involve combining existing measures into additional derived measures to provide more insight into the overall efficiencies and effectiveness at a project or organization level. The analysis at the organization level may be used to study productivity, improve efficiencies, and increase throughput across projects in the organization.
Tip
This paragraph describes in a simplified way how the organization’s process performance baselines (referred to here as “expected process performance”) support QPM that, in turn, provides the data used to refine these baselines.
The expected process performance can be used in establishing the project’s quality and process-performance objectives and can be used as a baseline against which actual project performance can be compared. This information is used to quantitatively manage the project. Each quantitatively managed project, in turn, provides actual performance results that become a part of baseline data for organizational process assets.
The acquirer may use quality and process-performance objectives to define performance and service level expectations for suppliers.
Process-performance models are used to represent past and current process performance and to predict future results of the process. For example, the latent defects in the delivered product can be predicted using measurements of defects identified during product verification activities.
The same measures of latent defects, analyzed using a supplier’s past projects data, can be used to predict the quality of products delivered by that supplier. The acquirer can use supplier process-performance models to predict the overall capability of the acquirer to deliver the product.
X-Ref
See the Brad Clark and Dave Zubrow presentation, “How Good Is the Software: A Review of Defect Prediction Techniques,” from the SEPG 2001 conference (www.sei.cmu.edu/sema/pdf/defect-prediction-techniques.pdf). Some of these techniques can be the basis for process performance models.
When the organization has measures, data, and analytical techniques for critical process, product, and service characteristics, it is able to do the following.
• Determine whether processes are behaving consistently or have stable trends (i.e., are predictable).
• Identify processes in which performance is within natural bounds that are consistent across process implementation teams.
• Establish criteria for identifying whether a process or subprocess should be statistically managed, and determine pertinent measures and analytical techniques to be used in such management.
• Identify processes that show unusual (e.g., sporadic or unpredictable) behavior.
• Identify aspects of processes that can be improved in the organization’s set of standard processes.
• Identify the implementation of a process that performs best.
• Identify aspects of processes that could be improved across acquirer–supplier interfaces.
Refer to the Quantitative Project Management process area for more information about the use of process-performance baselines and models.
Refer to the Measurement and Analysis process area for more information about specifying measures and collecting and analyzing data.
Refer to the Organizational Process Definition process area for more information about establishing organizational process assets.
Tip
Much of what is described in OPP (and in QPM) utilizes the practices described in MA. Mastering many of the practices in MA is a prerequisite to implementing OPP and QPM.
Specific Goal and Practice Summary
SG 1 Establish Performance Baselines and Models
SP 1.1 Select Processes
SP 1.2 Establish Process-Performance Measures
SP 1.3 Establish Quality and Process-Performance Objectives
SP 1.4 Establish Process-Performance Baselines
SP 1.5 Establish Process-Performance Models
X-Ref
A 2001 presentation made by Doug Smith and Craig Hollenbach at the CMMI Technology Conference and User Group (www.dtic.mil/ndia/2001cmmi/hollenbach.pdf) provides examples of process performance baselines and models.
Baselines and models, which characterize the expected process performance of the organization’s set of standard processes, are established and maintained.
Prior to establishing process-performance baselines and models, it is necessary to determine which processes are suitable to be measured (the Select Processes specific practice), which measures are useful for determining process performance (the Establish Process-Performance Measures specific practice), and the quality and process-performance objectives for those processes (the Establish Quality and Process-Performance Objectives specific practice).
These specific practices are often interrelated and may need to be performed concurrently to select appropriate processes, measures, and quality and process-performance objectives. Often, the selection of one process, measure, or objective will constrain the selection of the others. For example, if a certain process is selected, the measures and objectives for that process may be constrained by the process itself.
Select processes or subprocesses in the organization’s set of standard processes to be included in the organization’s process-performance analyses.
Hint
Select process elements, not processes composed of multiple elements. Multiple sources of variation hidden in a process are not easily analyzed. What might appear to be a stable process may in fact be a collection of unstable process elements. This collection of process elements would leave you unable to correctly predict future behavior or accurately identify opportunities for improvement.
Refer to the Organizational Process Definition process area for more information about the structure of organizational process assets.
The organization’s set of standard processes consists of a set of standard processes that, in turn, are composed of subprocesses.
Typically, it will not be possible, useful, or economically justifiable to apply statistical management techniques to all processes or subprocesses of the organization’s set of standard processes. Selection of processes or subprocesses is based on the needs and objectives of both the organization and its projects.
The selection of subprocesses for analysis, the determination of process-performance objectives, and the selection of appropriate measures are often concurrent and iterative processes of both the organization and its projects.
When selecting processes or subprocesses for analyses, it is critical to understand the relationships between different processes and subprocesses and their impact on the acquirer and supplier delivering the product specified by the customer. Such an approach helps to ensure that quantitative and statistical management are applied where they have the most overall value to the organization.
Tip
These examples of criteria used to select subprocesses are fairly sophisticated and are typical of organizations almost at maturity level 4 or 5. At lower maturity levels, only a few of the criteria may be practical. The criteria used will grow in sophistication and the selection evolves as more experience and data are gained.
The existence of project data that indicates the process or subprocess has been or can be stabilized is a useful criterion that can be used for selecting a process or subprocess.
Hint
Because not all processes contribute equally to a business objective, and all such analyses consume time, effort, and money, you should start with a small selection. As you learn more, you can modify and expand the selection accordingly.
Typical Work Products
- List of processes or subprocesses identified for process-performance analyses
Establish and maintain definitions of measures to be included in the organization’s process-performance analyses.
Refer to the Measurement and Analysis process area for more information about specifying measures.
Typical Work Products
- Definitions of selected measures of process performance
Hint
Analyze business objectives to identify process measures that provide insight into quality and process performance.
Subpractices
- Determine which of the organization’s business objectives for quality and process performance should be addressed by the measures.
- Select measures that provide appropriate insight into the organization’s quality and process performance.
The measurement repository provides common measures for this purpose.
For business objectives addressed through acquisition, select process, product, and service level measures that provide insight into the process performance of suppliers and into the quality of their deliverables.
X-Ref
The Goal Question Metric (GQM) is a well-known approach to deriving measures that provide insight into issues of interest. See www.cs.umd.edu/~mvz/handouts/gqm.pdf. The SEI’s variant of GQM is called the Goal Question Indicator Metric (GQIM). See www.sei.cmu.edu/products/courses/implement.goal-driven.sw.meas.html.
- Incorporate selected measures into the organization’s set of common measures.
Measures expected to be collected and reported by suppliers are incorporated into standard supplier agreement templates and standard service level agreements, as appropriate.
X-Ref
To begin systematic collection of these measures from new projects, incorporate them into the organization’s set of common measures (OPD SP 1.4).
- Revise the set of measures as necessary.
Measures are periodically evaluated for their continued usefulness and ability to indicate process effectiveness.
Establish and maintain the organization’s quantitative objectives for quality and process performance.
The organization’s quality and process-performance objectives should have the following attributes:
• Based on the organization’s business objectives
• Based on the past performance of projects
• Gauges process performance in areas such as product quality, productivity, cycle time, or response time
• Accounts for the inherent variability or natural bounds of the selected process or subprocess
• Accounts for the inherent variability or natural bounds of supplier performance
Tip
Objectives based on the organization’s business objectives may set the bar too high to motivate projects to identify process improvements. From a practical standpoint, what does the performance data show about how well a project can do relative to a particular process? In summary, there is need for balance between “desires” and “reality.”
Typical Work Products
- The organization’s quality and process-performance objectives
- Supplier service levels based on quality and process-performance objectives
Subpractices
- Review the organization’s business objectives related to quality and process performance.
- Define the organization’s quantitative objectives for quality and process performance.
Tip
The quantitative objectives for quality and process performance should be related to the organization’s business objectives.
Objectives may be established for process or subprocess measurements (e.g., effort, cycle time, and defect removal effectiveness) as well as for product measurements (e.g., reliability and defect density) and service measurements (e.g., capacity and response times) as appropriate.
- Define the priorities of the organization’s objectives for quality and process performance.
- Review, negotiate, and obtain commitment to the organization’s quality and process-performance objectives and their priorities from relevant stakeholders.
Tip
Commitment to the organization’s quality and process-performance objectives means senior management supports them by periodically reviewing how well projects are performing relative to them. Project management and senior technical staff members incorporate them into their projects and strive hard to achieve them.
- Revise the organization’s quantitative objectives for quality and process performance as necessary.
Establish and maintain the organization’s process-performance baselines.
Tip
The term process-performance baseline is used instead of process capability baseline because process capability assumes the events are generated from the same process (and people). This may be a correct assumption in the case of a single project (or team), but not for the whole organization.
The organization’s process-performance baselines are a measurement of performance for the organization’s set of standard processes at various levels of detail, as appropriate. The processes include the following:
• Sequence of connected processes
• Processes that cover the entire life of the project
• Processes for developing individual work products
There may be several process-performance baselines to characterize performance for subgroups of the organization.
Tailoring of the organization’s set of standard processes may significantly affect the comparability of data for inclusion in process-performance baselines. Effects of tailoring should be considered in establishing baselines. Depending on the tailoring allowed, separate performance baselines may exist for each type of tailoring.
Refer to the Quantitative Project Management process area for more information about the use of process-performance baselines.
Typical Work Products
- Baseline data on the organization’s process performance
Hint
Record sufficient contextual information with a measurement to enable identification of the process-performance baseline it should be included in, when it was generated, and by whom.
Subpractices
- Collect measurements from the organization’s projects.
The process or subprocess in use when the measurement was taken is recorded to enable appropriate use later.
Refer to the Measurement and Analysis process area for more information about collecting and analyzing data.
- Establish and maintain the organization’s process-performance baselines from collected measurements and analyses.
Tip
Unless the process is stable, process-performance baselines will actually be a mixture of measurements taken from different processes. Such baselines are severely limited in their usefulness to projects (e.g., trial natural bounds are likely to be far apart).
Refer to the Measurement and Analysis process area for more information about establishing objectives for measurement and analysis, specifying measures and analyses to be performed, obtaining and analyzing measures, and reporting results.
Process-performance baselines are derived by analyzing collected measurements to establish a distribution or range of results that characterize the expected performance for selected processes or subprocesses when used on a project in the organization.
The measurements from stable subprocesses in projects should be used when possible; other data may not be reliable.
Hint
Investigate subgrouping when incorporating data from multiple projects (and teams) into the same process-performance baseline. Even if the process is stable in individual projects, it still might be executed sufficiently differently across projects to make establishment of a single baseline inappropriate.
- Review and get agreement with relevant stakeholders about the organization’s process-performance baselines.
- Make the organization’s process-performance information available across the organization in the organization’s measurement repository.
The organization’s process-performance baselines are used by projects to estimate the natural bounds for process performance.
- Compare the organization’s process-performance baselines to associated objectives.
- Revise the organization’s process-performance baselines as necessary.
Tip
OPP does not directly say what to do with the results of the comparison of the baseline to the objectives. Ideally, the objectives are attainable, but a stretch beyond the baseline. This comparison establishes feasible objectives. If the objectives are infeasible, revise them using CAR or OID to search for ways to improve performance.
Establish and maintain process-performance models for the organization’s set of standard processes.
Process-performance models are used to estimate or predict the value of a process-performance measure from the values of other process, product, and service measurements. These process-performance models typically use process and product measurements collected throughout the life of the project to estimate progress toward achieving objectives that cannot be measured until later in the project’s life.
Process-performance models are used to estimate or predict when to fund, hold, cancel, migrate, reengineer, or retire a project. Process-performance models allow the acquirer to synchronize processes with customer needs. The organization’s process-performance baselines provide quantitative data on those aspects of the projects and organization that can approximate the throughput potential of its processes. Focusing on these critical constraints, process-performance models allow the acquirer to predict how to best maximize the flow of work through projects and the organization.
Hint
Establish process-performance models that provide insight at different points in a project (e.g., at the end of each phase) to track progress.
Process-performance models are used as follows.
• The organization uses them for estimating, analyzing, and predicting the process performance associated with processes in and changes to the organization’s set of standard processes.
• The organization uses them to assess the (potential) return on investment for process improvement activities.
• Projects use them for estimating, analyzing, and predicting the process performance of their defined processes.
• Projects use them for selecting processes or subprocesses for use.
Process-performance models are also used to set process-performance objectives for suppliers and to provide data that can help suppliers achieve these objectives.
These measures and models are defined to provide insight into and to provide the ability to predict critical process and product characteristics that are relevant to business value.
Results of the acquirer’s process-performance models are shared with suppliers to help ensure the synchronized delivery of products and services.
X-Ref
A paper by Tobias Häberlein, located at http://prosim.pdx.edu/prosim2003/paper/prosim03_haeberlein.pdf, concerns the application of system dynamics to acquisition. (System dynamics models were first used to model software.) The paper also has a good collection of references.
Refer to the Quantitative Project Management process area for more information about the use of process-performance models.
Typical Work Products
- Process-performance models
- Supplier process-performance models
Tip
A review of the organization’s set of standard processes helps to identify process and product characteristics that might assist in constructing a process-performance model. Process-performance baselines provide a primary source of the information needed to quantify and calibrate the model.
Subpractices
- Establish process-performance models based on the organization’s set of standard processes and the organization’s process-performance baselines.
- Calibrate process-performance models based on the organization’s past results and current needs.
- Review process-performance models and get agreement with relevant stakeholders.
- Support the projects’ use of the process-performance models.
- Revise process-performance models as necessary.
Hint
Meet with relevant stakeholders to discuss process-performance models (e.g., their usefulness and limitations) and the support required to make effective use of such models on projects.
X-Ref
To use process-performance models effectively, project staff members and management may need significant support. The subpractices of OID SP 2.2 provide some example forms of support.