Measuring Project Management Process and Practice Maturity

The best practices, tools, templates, and processes described in this section have been integrated to design a system for measuring process and practice maturity. This system is the major input to the CPIM described later in this chapter.

The Process Quality Matrix and Zone Map

The Process Quality Matrix (PQM) and the Zone Map shown in Figure 15-1 are the key data collection tools for CPIM.

My version of the PQM consists of 12 columns and one row for each process or process step. Note here that the PQM is a very robust tool. Any process can be represented in the rows that define the PQM. In my consulting experiences the process steps for the client's systems development life cycle have been a common application.

In the example shown in Figure 15-1, the first 10 columns are the 10 prioritized critical factors (CF) that cause IT project failure as defined by the Standish Group in their report CHAOS 2010. This list should always reflect the latest CHAOS report. The list of CFs is usually the Standish Group–prioritized list of reasons for project failure, but there are other options, too. So these 10 columns of the PQM are robust, as is the number of columns. The 11^th column contains a correlation factor that is computed from the data in the first 10 columns of the PQM, which will be discussed later. The 12^th column will contain a maturity level factor computed from a maturity assessment. The rows define process steps (such as the steps in a systems development life cycle) or, as used in this process improvement program, the processes defined by the Project Management Institute (PMI) in their PMBOK. All of the contents of the PQM are user defined. It can be customized to meet any client requirements. I have had occasion to use it in a variety of contexts.

Figure 15-1: PQM and Zone Map

To complete the entries in columns 1–10, identify which processes are affected by which CFs, and mark each with a filled box as shown in Figure 15-2. This exercise is usually done once and used in repeated maturity assessments. The total number of CFs that affect performance of each process is entered in column 11. That number will range from 0 to 10: the higher the number, the stronger the relationship between the process and the CF. Column 12 will contain an assessed process maturity level, which, in this example, is a scale of 1–5 as measured by the Project Management Maturity Assessment (PMMA). Any numeric maturity assessment can be used to generate the data in column 12.

The numeric data in columns 11 and 12 is then transferred to a Zone Map as shown in Figure 15-3. The Zone Map is one of the basic tools used to drive the process improvement program. Column 11 identifies the Zone Map row, and column 12 identifies the Zone Map column. In the example shown in Figure 15-3, four processes (P35, P36, P40, and P44) were given a Level 3 maturity value and are related to five CFs. This is reflected by the P35, P36, P40, and P44 in the cell that lies at the intersection of the column labeled 3 and the row labeled 5 in the Zone Map.

Figure 15-2: A completed PQM

The Zone Map has three zones of interest. The boundaries of each zone are permanent. Beginning in the upper-left corner is Zone 1, the most critical zone. Any processes that fall in this zone should receive a high priority in the process improvement program. The focus of that process improvement program should be to revise the process so as to minimize the impact of the CFs that are related to that process. Even in Zone 1, the processes that are closest to the upper-left corner are the most critical among Zone 1 processes. So in this example, P06 Integrated Change Control has a maturity of 1.44 and is impacted by 9 CFs. It is in most need of process improvements to minimize the impact of those 9 CFs. P12, P03, and P07 processes are next in most need of process improvement. Generally processes with lower maturity levels should have a higher priority for process improvement programs than processes affected by the same number of CFs. There are a total of 28 processes that fall in Zone 1, and these should be dealt with before any process improvement efforts are expended on the processes that fall in Zone 2. Zone 2 identifies processes that should be monitored. This zone runs roughly diagonally through the Zone Map and should receive the next level of attention for improvement based on available time and resources. There are 13 processes that fall in Zone 2. Finally, Zone 3, which runs along the rightmost columns and bottom rows, would not be of interest in the process improvement program. There are only 3 processes that fall in Zone 3. By taking into account the process maturity level and the correlation factor, processes can be ranked according to the priority needs for improvement programs.

Figure 15-3: A completed Zone Map

Figure 15-4 tells a much more important story. This figure shows the Zone Map for the seven Integration Management processes. Integration Management consists of processes P01 through P07, and all of these processes fall in Zone 1. P06, P03, and P07 should be the highest priority processes for improvement programs. The same is true of Scope Management, which consists of processes P08 through P12. They also all fall in Zone 1. These two Knowledge Areas will get the high-priority attention of the CPIM. Figure 15-4 reflects this Integration Management information in a much more intuitive format.

Figure 15-4: Integration Management Zone Map

In addition to the Zone Map, I have developed a graphic tool to help identify Knowledge Areas and processes within Knowledge Areas for improvement initiatives. An example of this tool is shown in Figure 15-5.

This is an intuitive plot of the process maturity levels for each of the nine Knowledge Areas (the dotted line) and the distribution of practice level maturity data from a 20-person sample of project managers (the box-and-whisker icon). The data in Figure 15-5 is from an actual client assessment and would have been gathered using the PMMA. The box-and-whisker icon for each Knowledge Area summarizes the data for that Knowledge Area. The end points of the vertical lines are the extreme practice maturities as drawn from the sample of project managers. The box represents the inter-quartile range (the middle 50 percent of the data points). I use a rule that says if the inter-quartile range falls entirely below the process maturity level (as is the case for Integration and Scope in this example), the associated process should have a high priority for an improvement initiative. That configuration says that 75 percent of the observed practice maturity data lies below the process level maturity for that Knowledge Area. This is a serious situation and requires immediate attention. If the inter-quartile range lies entirely above the process maturity level (Quality and HR in this example), that process should be targeted as a place to look for potential best practices among the project managers in the sample. In this situation, 75 percent of the observed practice maturity data lies above the process level maturity. In most cases that I have seen, this suggests that the process is not very good and project managers have substituted their own processes. Again, this is serious and immediate attention is required.

Figure 15-5: Process and practice maturity level plots

What Process Has Been Defined So Far?

Thus far, this chapter has defined the tools needed to conduct the analytical foundation in preparation for the Assessment and Analysis Phase of the CPIM (as you will see in Figure 15-6 later in the chapter). The following sections sum up the process to this point.

Step 1: Define the Process

Define an initial PQM using the PMBOK processes and the 10 CFs from the 2010 Standish Group CHAOS Report. Under the optional guidance of a trained consultant, discuss each process with reference to its relationship to one or more of the CFs. Those relationships define the PQM, which is the basis for prioritizing process-improvement initiatives.

Step 2: Validate and Finalize the PQM

The initial PQM needs to meet the needs of the PSO organization. The PQM will be reviewed with the PSO management team and modified as appropriate.

Step 3: Establish Correlations

In collaboration with the PSO, establish the correlations between the 44 project management (PM) processes and each CF. The 44 PM processes need to be correlated with CFs. This exercise should be facilitated by a trained consultant.

Step 4: Establish Metrics

Establish metrics to measure process performance on a project-by-project basis. The PMMA scorecard that I developed will be used to measure the maturity level of each process as practiced on a project-by-project basis. The use of the scorecard at the project level can also be used to identify the PSO's best practices to further improve process quality.

Step 5: Assess Project Managers against the PMMA

Audit a representative sample of project managers using the PMMA. This is the data needed to establish the practice baseline of project management.

Step 6: Assess Maturity Levels

Assess current maturity levels for all 44 processes. Based on the results of Step 5 and a review of the current organization's processes, determine the current maturity level of all PM processes. This will become the baseline against which future process improvements can be benchmarked.

Step 7: Plot Results on the PQM Zone Map

Plot the results on the PQM Zone Map in order to identify processes in need of improvement. The assessed maturity levels of all 44 processes are automatically transcribed into a Zone Map. The Zone Map is a high-level graphic view of the general maturity level of the organization's PM process. At the next level of detail, produce the Knowledge Area plot previously shown in Figure 15-5. You might even display the process maturity levels within a Knowledge Area in the same box-and-whisker plot for further identification of potential process and practice improvement areas.