Chapter 15
Tools and Techniques
In this chapter, we provide more detailed procedures for several of the tools and techniques we discussed in earlier parts of the book, as well as a bibliography. Remember, these are selective, noninclusive lists. Just because something isn’t on our particular list doesn’t mean that it isn’t worth looking into. Our experiences have exposed us to a subset of the literature and events in the process improvement world, and that’s what’s reflected here. Like all of you, we don’t have infinite time to read and review every article, book, presentation, or report that is released about CMMI and process improvement. We have even fewer opportunities to try new approaches—although we try very hard to do so whenever we possibly can. But the references that we rely on in our work are the ones included here. The only certain thing about this list is that it will expand and change over time as our own understanding improves and we continue to learn nuances of different aspects of process improvement.
15.1 An Example of Setting Smart Goals
When an organization has established a set of SMART goals for its business, deriving process improvement goals is usually fairly obvious. Consider the following based on the Balanced Scorecard:
• Customer-quadrant business goal: Add at least two new customers to our portfolio.
An improvement goal associated with that business goal might be:
• Customer-quadrant improvement goal: Ensure that all our customer-facing processes are visibly responsive and cost-effective for existing customers.
The premise is that if our customers see responsive, cost-effective processes, they will recommend us to new customers.
But is this a SMART goal? Let’s look at each element.
Specific: The goal addresses customer-facing processes rather than all processes, and it’s looking at two specific attributes: “visibly responsive” and “cost-effective.” If we actually know which processes are customer facing, that could be sufficient, or if there are really too many customer-facing processes for us to deal with at once, we might want to change that to one that we know needs to be improved. We may also want to be more specific about existing customers. So maybe we’ll modify it to:
• Ensure that our customer requirements management process is visibly responsive and cost-effective for existing product development customers.
What about measurable? The two measurement-related aspects are “visibly responsive” and “cost-effective.” The first question here is, “Cost-effective for who? Customers? Or us? If for them, ‘cost-effective’ might mean that they never have to pay for a change after the requirements are agreed to… but that could be devastating for us if they change their minds all the time! And how do we measure ‘visibly responsive’? Maybe that means that they can always tell the status of any change that they have initiated with us?”
Let’s assume that this is what we mean. So the next mod might be:
• Ensure that our customer requirements management process allows customers to have 24-hour access to their requirements-change status and that this access doesn’t cost them anything additional.
Whoa! Did we really mean that “cost-effective” was related to status, or did we mean that our whole requirements management process is cost-effective?
If the former is the case, this goal probably works. If the latter, we probably have a different goal we need to create related to the cost-effectiveness of the overall requirements management process. Let’s say we decide to create a second goal, but we actually like the fact that our improvement goal for responsiveness has a cost element to it, so we’ll keep it too.
Is this goal achievable/attainable? At this point it probably is, although when it was more general, it may have been too vague to determine what the actions would be to achieve this goal. One action we might envision (not the only possibility) would be to establish an 800 number that is staffed 24 hours a day by someone who has access to our internal change database and who will answer questions for customers about their change. Another action might be to create a customer-accessible, password-protected Web site for customers to use. Thus the goal is actionable and no further changes are needed at this point.
Now let’s talk about realistic. We established the goal’s relevance when we said this was a way to attract new customers. Note that both of the above actions assume that we have an accessible requirements change database. If that’s true, one of the actions could be reasonable, and if we think we have the resources to do either one of them, we’d say that it’s a realistic goal. In the search for realism, we may also come up with some other actions/solutions that would be more reasonable than these two. If, on the other hand, our internal requirements-change process status is not available for access, this may not be a realistic goal for us yet. A related enabling goal might be to:
• Understand the status of any requirements change at any point in time, from its request through its completion and incorporation into a new release of the product.
If we need to pursue this derivative goal first, we should do the same SMART analysis on it and improve it to where we can use it.
But let’s assume that the first goal is still useful. The last thing to check on is time-basis, or tangibility. For this goal, a time basis is probably relevant. We already have one time-basis in the goal when we say that customers should have 24-hour access. The other aspect of time we should think about is when this needs to be done. Within the next week? Next month? Next year? Most business/improvement cycles do goal setting every year or so unless there’s some particular environmental pressure we have to respond to. So let’s change our goal to:
• Ensure that our customer requirements management process will allow our product development customers to have 24-hour access to their requirements-change status by September 2007, and that this access doesn’t cost them anything additional.
At this point, let’s compare our original goal to this one. Remember, it started out as:
• Ensure that our customer requirements management process is visibly responsive and cost-effective for existing customers
Which of these two would you rather sign up for if your bonus depended on it? We’d rather sign up for the revised goal if we agreed that it’s realistic (that is, we agree that we have the basis internally for doing this and will get the resources to create a solution that’s reasonable). Even if we didn’t think it was realistic, we could probably negotiate some parameter changes on this goal that would make us and our sponsors happy, because we can actually understand what’s being asked of us.
We would want to accept the original goal only if we would be the ones to establish all the success parameters. Otherwise, we could make some really great improvements, but the sponsor could say, “That’s not cost-effective for us, even though it’s cost-effective for our customers,” or any other number of things related to the ambiguity of this goal.
15.2 Performing a Cmmi-Based Business Analysis
To run a CMMI-based Business Analysis (CBA), first assemble the following:
• A facilitator who understands implementation of the model (probably a consultant who has worked in CMMI implementation successfully).
• Your sponsoring team and opinion leaders within the groups that are planning to adopt CMMI.
• A conference room with a good amount of wall space (enough room for at least ten flip-chart sheets hanging on the walls).
• Sticky notes. (We use 3-inch by 3-inch notes, for the most part; they give you enough room to write a phrase but not a paragraph.)
• Flip charts (preferably the stick-to-the-wall type).
• A representation of the model that provides just the Purpose, Goals, and Specific Practices for each Process Area you’ll be talking about; for example the appendixes in CMMI Distilled or the Mini CMMI available from www.cooliemon.com or Amazon.com in either staged or continuous representation.1
1. Williams, Ralph, and Patrick Wegerson. MINI CMMI (SE/SW/IPPD/SS Version 1.1) Continuous Representation. (Harmony, Penn.: Cooliemon LLC, 2002).
Before doing the analysis, you may want to meet with your facilitator to decide on an initial scope. Rather than take people through all 22 PAs, you may decide (as many people do) that just going through the Maturity Level 2 and 3 PAs is enough to get started. If you are working in an organization that has already been doing process improvement and you’re looking at “what’s next,” you may have a different subset of PAs that you want to work with for the analysis.
When you’ve decided on the scope and acquired the participants, perform the setup and execution procedures in Tables 15-1 and 15-2.
When you’ve completed this analysis, you should have a set of around three CMMI Process Areas to work.
Table 15-1: Instruction for CBA Room Setup
Table 15-2: Execution of a CBA
A few tips on selecting projects:
• For best timing, it’s best to match up projects with PAs that are relevant to the kinds of work the project is doing in the next few weeks. Assigning a project that is almost completed to work on the Project Planning Process Area might not be the best choice.
• If it’s convenient, having more than one project to work on the same PA can produce synergy among the projects and achieve some team building. If this is your first cycle through, don’t demand that the group come up with the identical solution for each project. There are likely to be environmental differences and team differences that drive some different, but equally valid, solution approaches. If you can get the projects to follow the same format for process guidance, that’s actually enough for the first time through.
• If you have a choice between a project with a short life cycle and one with a longer duration, choosing the shorter project is generally better; you will be able to see the results of your improvements faster and can determine whether you want to deploy further, or do more refinement before further deployment.
15.3 Performing a Readiness and Fit Analysis
To perform your own Readiness and Fit Analysis (RFA), you need to assemble the following:
• Someone who understands implementation of the model (probably a consultant who has worked in CMMI implementation successfully).
• Your sponsoring team and opinion leaders within the groups that are planning to adopt CMMI.
• A conference room with a good amount of wall space.
• Sticky notes. (We use 3-inch by 3-inch notes, for the most part; they give you enough room to write a phrase, but not a paragraph!)
• A facilitator who understands how to perform clustering or affinity diagramming.
• Flip charts (preferably the stick-to-the-wall type).
• Sufficient forms similar to Figure 15-1 to allow participants to record their scoring.
When you’ve gathered the participants and prepared the room, Table 15-3 provides the basic procedure.
Figure 15-1: RFA form
Table 15-3: Procedure for Performing Readiness and Fit Analysis
15.4 One-Hour Process Description Method
Our favorite technique for creating, integrating, and verifying/validating your process descriptions is called the One-Hour Process Description method. It’s aptly named because you need only about one hour of time from the subject-matter experts for the process you want to define. Of course, the team working on documenting the processes for improvement and future use will have a good bit more than an hour’s work to prepare and to follow up after the definition session.
The goals for this kind of workshop are to:
• Quickly create a first-level draft of as-is or to-be processes based on a defined process architecture that is ready to be transcribed for future review by stakeholders.
• Identify areas where process guidance exists or doesn’t exist for subprocess or tasks.
So what can you actually do in one hour? Experience shows that from two to eight three-person teams creating/reviewing their descriptions can create a 70-80 percent complete draft of a well-bounded life-cycle stage or support process. To accomplish this, ahead of the session, you will need to:
• Identify the process architecture (how you want the process descriptions broken into sections).
• Identify two to three subject-matter experts for each topic. This can become limiting on the process when you’re in a small organization, you have only one or two people who know multiple processes well, and very few others to draw on as subject-matter experts in even one process.
• Establish a candidate-roles list, with definitions for each role. (These are the role names you will use in the definition session.)
• Gather, and make available to the subject-matter experts, current process guidance artifacts to help jog their memories if they get stuck and to help solve disputes that may come up.
The general approach is to use two-to three-person teams of process “owners”/subject-matter experts to define processes they are knowledgeable about and to review processes in their customer/supplier chain that are being defined in the same session. This is a very low-tech technique; you just need flip charts or mural paper, multiple colors of sticky notes (different colors for Roles, Tasks, and Deliverables), and pens. There are three main roles in the process: the Facilitator, the Champion or Engineering Process Group Lead (the person sponsoring the event), and the Participants. Figure 15-2 is a summary diagram of the workshop.
This method is useful as the basis for a model-based “mini-appraisal” by someone who knows the reference model being used. It is also useful as a generator of a first draft as-is or to-be process description in a medium-size or larger organization. It is particularly useful where subject-matter experts are difficult to schedule for continuous periods of time.
This method is not as useful when you are at the stage of generating detailed procedures and job aids, or when subject-matter experts are not available. It is also not meant to provide rigorous appraisal results (for example, for a SCAMPI A against CMMI) or in the event that there is no process architecture available as guidance for breaking up the process into chunks.
Figure 15-2: One-Hour Process Description Summary Diagram
15.4.1 Critical Success Factors
Here are a few facilitation hints that make this technique more effective:
• Plenty of physical space should be available for team work.
• Use sticky notes on flip charts or mural paper to make it fast and easy to move things around during the definition session and force definers to keep it simple.
• Use three 15-minute description and 5-minute review cycles to keep the process description focused and relevant.
• Team members need to accept the 15-minute deadline.
• Team members should be assigned only to processes they understand.
• The Facilitator needs to be capable of spotting “too high/too low” descriptions and performing course correction on the fly.
• Descriptions produced in the workshop should be thoroughly reviewed by other stakeholders after the workshop.
People frequently ask “How many teams should we use?” The answer usually depends on how many appropriate process owners/subject-matter experts are available:
• If you have a small number of teams and/or a large number of processes, you will need several sessions.
• Even with just two teams, six processes can be described in one afternoon.
• If you have a large number of teams (and a large room), you can do as many as seven or eight processes in one session.
15.4.2 Workshop Preparation
So that you can minimize the impact on the participants’ schedules, there are several things to complete before you begin the workshop. You want to “hit the ground running” when the players arrive, instilling confidence in the participants that you are there to help, not waste their time.
Process “architecture”is needed to organize the description.
Before the workshop session, you need to decide on a process architecture, the high-level structure of the content that will be described. Most organizations start with a time-phased or life cycle-based architecture. If you’re getting started and don’t have an architecture that is easily identifiable, you’ll probably use the as-is process to get started. Here’s some simple architecture guidance:
• For as-is process descriptions:
Often, CMMI Process Areas or existing local life-cycle phases are selected.
• For to-be process descriptions:
Subprocesses should relate to the desired implementation focus, usually not Process Area-based.
Subprocesses should integrate with parent life cycles, where applicable.
The architecture will be used, at its highest level, to create the mini-teams.
Deciding on names for the roles in your organization.
Before the workshop, you also need to identify the major roles involved in the processes and decide on the names you will use for them in the workshop. This may sound like a trivial step, but if you do not do this, when you try to sequence the different subprocesses, you will spend a lot of unnecessary time resolving whether “project leader” and “project manager” are really the same role.
When you have a set of role names defined, send it out to the workshop participants for review and questions. If you can answer their role-based questions before the workshop, things will go more smoothly.
Preparing your “demo”subprocess.
The most effective way to get participants comfortable with the process and speed of the description task is to demonstrate it by using one of the simpler processes, preferably one that most of the workshop attendees participate in. If you don’t have any other ideas, Requirements Management is usually a good one.
Create a table on a flip chart with role sticky notes across the top or down the side. If roles are across the top, your task flow will be top to bottom. If roles are down the side, your task flow will be left to right. SuZ’s preference is for roles to go down the side; that’s probably because her knees are starting to object to working down by the floor as she gets to the end of a process.
Assuming that you have put your roles down the side, fill in task sticky notes by the primary role(s) involved in the task, along with the work products or events that result from the task. If another role is involved in the task at that time, give it its own sticky note. If there’s a dependency relationship, draw an arrow from the task providing the input to the task producing the output. Continue adding sticky notes and workflow arrows.
When you’re satisfied with your description, copy it (through a digital photograph or the old-fashioned pencil-and-paper method) on a 8½ Χ 11” or A4 sheet of paper. Set up a second flip chart with just the roles and no tasks, deliverables, or arrows. When you do the demonstration, do it with this empty flip chart. Demonstrate two or three tasks, and when you think the participants have the idea, bring out the finished flip chart(s) to show them the level of granularity you’re looking for.
15.4.3 Basic Process For Workshop
The following steps describe the flow of the workshop.
Entry criteria for workshop
• Room is prepared.
• Mini-teams and processes are selected.
• Role names are defined and sent out to the mini-team participants.
• Existing process artifacts are gathered and available in the room for mini-teams to use.
• Demonstration materials have been prepared.
Approach for process descriptions
Form teams related to process topics. (Use local process architecture as a guide.)
Each team uses color-coded sticky notes to identify the major:
• Tasks
• Roles
• Deliverables (work product or event)
As an option, teams may also be asked to identify (with a different-color sticky note, of course) process guidance available related to their assigned process/subprocess.
Assigned review teams “switch” with the primary team to review evolving drafts, based on a customer–supplier chain. Figure 15-3 Illustrates the Recommended Cycles.
Figure 15-3: Process Description Cycles
Ground rules for process description
This is a set of ground rules that we give each mini-team:
• Use one task, role, or deliverable per sticky note.
• Make sure that you look at your assigned process throughout the entire product life cycle (initiation > retirement). Generally, use one table or flip chart per life-cycle phase unless the activity during a phase is really minor. (This is most relevant for processes, such as Configuration Management, that have different activities that are expected to occur at different times in the life cycle.)
• Keep it simple!
• If capturing “as-is,”
Post ONLY current practice (do capture ideas for the future elsewhere).
You may want to capture “barriers to this step.”
• Use a maximum of ten tasks per subprocess.
• If you’re in doubt as to level of granularity or other such questions, don’t hesitate to ask the facilitator.
Incorporating process guidance into your process description
If you’re exercising this option, chances are that you will go over the one-hour mark. (Don’t extend the work sessions longer than 15 minutes; instead, add another session.) This step could also be done after the main workshop by a smaller team that knows what process guidance is and is not available.
Assign different colors to small sticky notes (we usually use 1 Χ 2” notes) to represent “guidance available” and “no/inadequate guidance.”
For tasks or work products that clearly have current process guidance:
• Use “guidance available” sticky-note color.
For tasks or work products you’re unsure about:
• Start with “no guidance” sticky-note color.
• The Facilitator reviews to identify documentation that may be missing.
Don’t obsess! If you’re unsure, mark what you think the answer is, but put a question mark on that sticky note so you know to review it with someone who knows the guidance better.
You’re done, but you’re just getting started!
If you are the Facilitator, you have several important postworkshop tasks:
• One minor but important detail is to number all the sticky notes before taking down the flip charts so that any sticky notes that fall down can be replaced. A good way to do this is to use a convention we call RNX, where R= Role, N= subprocess, and X = task. We also tend to use clear tape to do a gross “tape down” of the sticky notes on the flip charts or mural paper. It is also useful to circle each task cluster and number the clusters so that they can be re-created from sticky notes that fall down.
• Transcribe flip charts into diagrams or tables, using your preferred process-visualization tool. (Microsoft Visio has a nice template that we often use; sometimes we just use a Microsoft Word table.)
• When the materials are transcribed, review them with the participants to make sure nothing went awry and to allow them to refine their descriptions based on what they’ve thought about in the meantime.
• When the participants are satisfied with the descriptions, bring both the transcribed files and the flip charts to scheduled review meetings with the different roles called out in a particular subprocess and with other stakeholders you know will add value. Reviewing the flip charts themselves is very interactive and best for a “meeting”-type review rather than an e-mail round robin. Keep in mind that the reviews are highly likely to take longer than the original draft.
• During the reviews, look for “must include/must not include” types of conflicts. Translation: Some people will say, “It’s not correct unless you include x” or, conversely, “It’s correct only if you delete x.” This gets thorny though when one role says a task must be included while another says it must not be included. If these kinds of conflicts come up in the review-session meeting, commit to take the parties in the conflict offline to resolve the issue. If you don’t get the issue resolved, your description will lose credibility with those who are relying on your accuracy, and when it comes time to derive the to-be process, you are likely to get lessthan-stellar participation.
Next steps
The next steps you engage in depend on your original goal:
• For mini-appraisal, CMMI gap analysis can be performed after the initial session, even before stakeholder review.
• For an as-is process, desired improvements to the process are identified.
• For a to-be process, work packages of process guidance can be formulated for assignment or production.
15.5 Infusion and Diffusion Measurement
In Chapter 9, we introduced the concept of adoption progress measurement. This type of measurement attempts to provide indicators of progress toward your adoption goals for CMMI (or any other technology you want to apply it to). We wish we could tell you that there are “standard” indicators for measuring adoption progress, but so far we haven’t found any. The ones we present here have been used in pilot settings but should still be considered somewhat experimental. The work to improve and evolve them is continuing, although at a slow pace. In particular, the diffusion measurements exhibit a “break” in the type of measurement that, from a researcher’s viewpoint, makes them irregular. (You move from measuring numbers of individuals completing events to measuring existence of particular organizational artifacts/practices.) So far, this hasn’t created much of a problem in practice, but we hope that in future editions, we’ll have more evolved indicators for you. In any case, the premise that you need to measure adoption progress before you can know when it’s “safe” to calculate ROI is a sound one, both from a research and a field perspective. Even if you don’t find our approach useful, we hope you’ll take the time to decide on some indicators of your own.
15.5.1 Infusion Measurement
Infusion, as illustrated in Figure 15-4, defines the “depth” of the adoption:
• How integrated are the new practices into the existing work practices and culture of the organization?
• How integrated should they be?
• Which roles need to be have a “deeper” Level of Use (and, therefore, understanding) than others?
Establishing Level of Use goals is one way to establish measurements for gauging infusion. In addition to helping with understanding your adoption success, establishing Level of Use goals helps you scope the need for training and other transition mechanisms needed to support adoption for different subsets of your adoption population.
The first thing to think about in defining Level of Use goals is what roles are critical for adoption of these practices. When you are deploying a new process, be it a new project planning process, a new services integration process, or a new proposal process, it is useful to ask, “What are the different roles that are likely to be involved?” Typical roles for process deployment include someone who will
• be in charge of developing the process
• control changes to the process
• use all the aspects of the process
• use only one aspect or some aspects of the process
• manage those who use the process
When you have an idea of the roles that need to be accounted for in the development and deployment of the process, think about what level of understanding/use of the process is necessary for each role’s success. Think in terms of the needed understanding of the improvement process as well as the process content. For example, the process developer typically needs more depth of understanding of the PI activities than an end user of the process. This typically varies from broad and deep knowledge to narrow and cursory. In this context, understanding CMMI or another model’s content would be a “specialty” knowledge area just for those developing the process to be seen by others.
Figure 15-4: Conceptual View of Infusion
Next, think in terms of how many people are in each role group that you’ve defined. The number of people may impact training or communication mechanisms implementation choices you make based on the level of understanding you have specified. If only one or two people are in the Process Developer role, for example, and they need to understand CMMI deeply, you may want them to attend an SEI CMMI course. It will probably be cheaper to send the two of them to a public course than to bring in an on-site course designed for 24 people who don’t really need that depth of knowledge about the model.
Once Level of Use goals are established, you need to establish criteria for accomplishment of that Level of Use. (Essentially, you’re creating an explicit success condition.) Many of the same events you’re tracking for diffusion measurement will be part of the criteria for Level of Use. For an end-user role, for example, there may be a general orientation course required, followed by a specific Web presentation on using the new process. In addition, there may be a checklist that records the results of process completion to be turned in when the process is completed. When two checklists (or three, or however many you think establish regular use of the process) have been completed, that end user is considered to have met his or her Level of Use goals. This information will also be helpful in tracking diffusion of the process.
When you have criteria for each goal, and you know which of your staff fit into each role category, you can track achievement against the criteria you’ve established. Figure 15-5 illustrates a profile as one way of tracking. Note that not all use/role categories have lots of people who are part of that category.
Figure 15-5: Example Level of Use Profile
For some categories, you may need or want only one or two people to be in a usage category.
In Table 15-4, we provide an example set of Roles and their respective Level of Use understanding goals. From this table, you could derive the criteria that would confirm that role has met the understanding and usage goals. Note that you are likely to have different names for these roles within your own organization and that the actual usage goals may differ for different settings. In some settings, for example, the Process Developer may also be required to be a User “Doer,” so he or she would be expected to use the processes he or she develops. In other organizations, those roles may not be combined.
The following is a suggested set of Level of Use categories for process improvement:
Casual 1. Casual use of a single, or only a couple, of the processes being improved
Casual 2. Casual use of many of the processes being improved
Moderate 1. Moderate use (probably would be in the approval cycle related to at least one process) of one or two processes being improved
Moderate 2. Moderate use of several of the processes being improved
Heavy 1. Heavy, probably daily, use of one or two processes being improved
Heavy 2. Heavy, probably daily, use of several of the processes being improved
Expert 1. Subject-matter expert, capable of defining and improving one or two of the processes being improved
Expert 2. Subject-matter expert capable of defining and improving several of the processes being improved
Expert 3. Versed in process improvement techniques and a process subject-matter expert.
Expert 4. Expert in process improvement techniques
When you’ve established Level of Use goals, you can derive the transition mechanisms that will be needed to support achieving them and add them to your infrastructure planning.
15.5.2 Diffusion Measurement
Diffusion, as illustrated in Figure 15-6, measures how broadly the process improvement has been adopted by the organization. The approach documented here is an expansion of an idea from Kim Caputo’s book CMM Implementation Guide: Choreographing SW Process Improvement.2 Even though Kim focuses on her experience of SW-CMM adoption at Unisys, most of what she talks about is easily adapted to CMMI, so it’s worth reading. She uses a choreography metaphor throughout the book to illustrate her points. Some people don’t relate well to that metaphor, but the content of the book is good enough that it’s worth reading even if you skip the choreography parts.
2. Caputo, Kim. CMM Implementation Guide: Choreographing Software Process Improvement. (Reading, Mass.: Addison-Wesley, 1998).
As you might guess, infusion and diffusion are independent measures. Diffusion measures are most useful in larger populations. In smaller settings, you can probably assess diffusion informally. However, you’ll still want to establish and use more formal Level of Use (infusion) goals.
Table 15-4: Example Roles and Levels of Use
To measure diffusion, first you have to identify key transition mechanisms for your process improvement effort. Then you can use the concept of phased transition mechanisms to help build a profile of adoption progress:
• Define the key events that constitute evidence of movement from one commitment state to another.
Figure 15-6: Conceptual View of Diffusion
• Create measures that allow you to know when those events have occurred.
• Gather and chart the measurements.
The example that follows provides “notional” profiles as an organization progresses through a process improvement effort. In the following charts, on the X axis, C/A stands for Contact/Awareness, U stands for Understanding, TU stands for Trial Use, A stands for Adoption, and I stands for Institutionalization.
Let’s say that the main Contact and Awareness event being measured is attendance at an all-hands meeting where the process improvement effort is being announced and introduced. When everyone has attended that meeting or has read the presentation as a makeup if he or she was absent, your profile would look like Figure 15-7. One hundred percent of the relevant staff have completed that event.
If you’ve defined Understanding events as a set of role-based training events that people would be expected to attend and Trial Use as participating in a pilot, you would collect information on who has attended those classes and who has participated in pilots, and add those numbers. At some defined point, maybe three months from the initial meeting, you look at the numbers again and get a different profile (Figure 15-8). About 25 percent of your population has moved from Contact/Awareness to Understanding or Trial Use. Questions that a graph like this does not answer are “Is this fast enough?” and “Are the right people attending the training classes so that we progress as expected with the pilots?” If you’ve established Levels of Use goals and are tracking data for those, you may already be able to answer these questions. When you’ve gone through the improvement cycle a couple of times, you can set targets for different profiles at different points in time as you go through your improvement activities. Initially, it’s usually enough just to look at the actual data and start asking questions.
Figure 15-7: After the All-Hands Meeting
Figure 15-8: Basic Classes for Key Roles Held
In Figure 15-9, we’re seeing that more people are attending the classes and participating in the pilots. Depending on when this snapshot was taken, this may be good progress, or it may be slower than anticipated. If you think that progress toward adoption is slower than it should be, it’s a good idea to ask why. Why aren’t more people attending classes? Why aren’t more people participating in pilots? The answers to those questions could highlight barriers you weren’t aware of before that need to be removed.
In Figure 15-10, we’re starting to see some of the population move into Adoption and Institutionalization. The measures for Adoption might include how many employees have been through the new process more than once, or how many employees have responded to a project-retrospective questionnaire that gathers data about the new process. Some of the other measures you might use for Adoption are not so much about the number of employees as about the existence and use of particular mechanisms by defined audiences. One measure of Adoption, for example, might be the use of a particular set of project review questions by Project Managers. In this case, it’s not the entire population you’re after, but it’s more a Level of Use measure that’s also used to support diffusion measures. For Institutionalization, the measures tend to be around the existence and use of certain policies and specific types of training (for example, new-employee orientation). This conflation of different types of measures makes this more of a heuristic measurement.
Figure 15-9: Several Pilots Have Started
Figure 15-10: Widespread Adoption has Begun
In Figure 15-11, we’re seeing more people completing the Understanding training events, more people using the improved processes for the first time, and more people using the processes routinely, as well as the appearance of some of the indicators that we’re institutionalizing the processes. This is probably the first point in our example where asking the question, “What’s our ROI for this change?” would make sense, although you would probably want to see more people actually using the processes (at least 40 percent) before seriously looking at ROI. Why 40 percent? In Everett Rogers’s diffusion research,3 37 percent was the point at which self-sustaining adoption was seen to occur, so we use 40 percent as our heuristic.
3. Rogers, E. Diffusion of Innovations. (New York: The Free Press, 1995).
In Figure 15-12, we’re definitely at a place where calculating ROI would be a reasonable thing to do, provided you have actually measured some of the things we suggest in our section on business value. There are still some laggards who haven’t finished training or started using the processes, but it’s hard to tell just from this chart whether it’s because they don’t want to use the process or whether they may be part of the population that doesn’t encounter these processes very often. You can see from this how you can make a better interpretation of the diffusion information if you also have some infusion information to pair it with.
Figure 15-11: Starting to See Institutionalization
Figure 15-12: Moving into Widespread Use
Figure 15-13: Improvement is the New Status Quo!
In Figure 15-13, the processes should be firmly in place and the risk of back-sliding should be relatively low. You will always have some percentage of your population moving through the earlier stages of adopter commitment if you’re experiencing any kind of growth or turnover, because your population is likely to be expanding. We would never expect to see a graph that showed only populations in Adoption and Institutionalization unless the population were static or declining.
We’ve already mentioned that the character of diffusion measures changes as you move from Trial Use into Adoption and Institutionalization. Here’s a bit of guidance on selecting measures.
For Contact, Awareness, Understanding, and Trial Use:
• You typically count the numbers of the target population who have attended some event or received some communication.
For Adoption and Institutionalization:
• You typically count the existence or absence of specific artifacts, such as a policy, that affects a subset of the population or the use of a particular mechanism, such as measurements, with a specific subset of the population (for example, senior management).
Note that because you’re counting percentages of the population, you don’t automatically know from this graph the actual numbers of people going through the process. You may find it useful to graph actual numbers; however, if you’re comparing different parts of an organization, the percentages may give you a better view of the diffusion progress, even though the sizes of the different parts of the organization are different.
It is very important to remember that this type of data collection can be overkill for very small organizations. If you have only 20 people affected by a change, it’s pretty easy to see who’s missing from the training events and who is actually using the new processes, and this kind of explicit diffusion measurement would not be as necessary. It’s most useful in a larger organization, where it’s more difficult to have a broad view of the actual participation in different improvement activities designed to promote adoption of the new processes.
15.5.3 Establishing Events for Adoption Diffusion Measurement
Table 15-5 provides more detail on exit criteria for the Adoption Commitment Curve categories covered in Chapter 13 and suggests some events you might consider as measurable events for your diffusion profile.
Table 15-5: Diffusion Events Suggestions
Table 15-6 provides a template for recording your own diffusion events. One possible use for this template is to have all the members of the improvement team fill it out separately, and then look at the responses as a group and discuss the similarities and differences in the events that were chosen. A good question to get people going is, “What kinds of events or evidence would be appropriate for a user of the new processes to go through to transit each stage of the commitment curve?”
Table 15-6: Blank Diffusion Events Template
15.6 CSI (Crime Scene Investigation) Technique + Chaos Cocktail Party
This section combines two techniques that work very well together: the CSI (Crime Scene Investigation) and the Chaos Cocktail Party. SuZ learned both of these techniques at a NASAGA (North American Simulation and Gaming Association) conference a few years ago and has had generally wonderful responses to both techniques.
The purpose of the CSI exercise is to get people to think beyond symptoms to root causes. The device of a taped-out “body” is used to represent whatever failure is being analyzed. The way we usually frame the question to think about is “Why did X [X is the topic of interest] die?” So you could have “Why did the process improvement effort die?”, “Why did the training event die?”, and so on. Participants are asked to review the “evidence” (sticky notes with the symptoms being observed in the organization) and think about the possible root causes for the “death.” CSI can be used with a small or large group. If the group is larger than 20 people, you may want to tape out two bodies so that people can actually see the sticky notes. Figure 15-14 shows an example of a “body” set up for this exercise.
When people have written their root causes, ask each participant to exchange his or her index card with someone else in the group. On the back of the new index card, each participant writes his or her preferred solution to the identified root cause. We also instruct them to put a line through the root cause so that it’s easy to tell which one is intended as a cause versus which one is intended as a solution. Then you can use any prioritization scheme you like to discuss and prioritize the solutions. Our favorite follow-on at this point is the Chaos Cocktail Party.
The purpose of the Chaos Cocktail Party is to actively and quickly prioritize a large number of ideas without losing broad visibility into the ideas. Use index cards to get people’s answer to the focus question and then give them brief amounts of time (around 30 seconds) to exchange their index cards with others in the group. Each time they exchange, they read the card and then exchange the card with someone else for a different card. At about 30-second intervals, the facilitator stops the exchange, and people pair up to do a forced binary rating of the two cards. The number of rounds of exchange/rate depend on the number of participants—usually three or five. After the exchange rounds, participants add up the score for each card, and the facilitator counts down scores and collects the highest-scoring items for further discussion.
Figure 15-14: Example “Body” for CSI Exercise
15.6.1 CSI (Crime Scene Investigation) Example Instructions for Participants: Why the Training Module Died
1. Get one index card from the back table, one of the instructors, or one of the room monitors.
2. Get up and go to one of the crime scenes. Take your index card with you.
3. Look at the clues. They’re all related to “why the training module died.”
4. Write on the index card the one possible “cause of death” for the training module that you think is most important to investigate.
5. Exchange your cause of death with someone else.
6. On the opposite side of the card you received, write what you think is the one most promising solution to prevent that cause of death.
15.6.2 Chaos Cocktail Party: Follow-on from CSI
1. Cross out the cause of death on your index card.
2. From now on, focus only on the solution side.
3. Repeat the following exchange-review cycle three to five times (the facilitator will tell you how many):
4. Exchange-review cycle:
a. When the instructor calls time, exchange cards as many times as possible with others, glancing at the solution but not discussing it.
b. When the instructor calls time, find a partner, and look at the two prevention actions (one on each of your current cards).
c. Together, allocate seven points between the two cards according to your opinion as to the value of each solution and write the values on the appropriate cards.
5. Tally the score of the index card you were left holding at the end of the last round.
6. The instructor will call out scores, going down from 35 (if five rounds were used), asking for items that fall in that score range.
7. The top five to ten items go on flip charts for discussion.
The cartoons shown in Figure 15-15 can also be used in conjunction with a CSI exercise; they were prepared by Shawn Presson for our tutorial on active learning, and they provide a bit of dark humor for a group that is dealing with a failure-analysis process.4 Think of this as being like the U.S. Army Survival Manual metaphor; get participants to think up PI-relevant analogies to the causes shown here.
4. Garcia, Suzanne, and Shawn Presson. “Beyond Death by Slides.” In Proceedings of the Software Engineering Process Group Conference 2004. (Pittsburgh: Carnegie Mellon University, 2004).
Figure 15-15: Archetypal Death Cartoons (Adapted from Garcia and Presson’s “Beyond Death by Slides” Tutorial)
15.7 Additional Resources
In this section, we list some additional resources you may find interesting. Some are mentioned in earlier chapters, but we thought that having them in a single place might help you explore a bit more.
15.7.1 Conferences of Interest
One of the best places to learn about what process improvement techniques really work and how to approach real-world problems is at a conference. You can attend sessions or just network informally. We list some of the betterknown conferences that address process improvement issues.
SEPG (System and Software Engineering Process Group) conferences
SEPG conferences are currently held annually in four locations: United States (usually March), Europe (usually June), Australia (usually September), and Latin America (usually November; primarily Spanish and Portuguese-language submissions). All conferences are co-sponsored by the SEI and one or more partners.
These conferences are among the best sources of information on implementation issues and solutions for model-based process improvement, and they’re great places to network with like-minded individuals who are facing problems similar to yours. The largest, in the United States, typically hosts upward of 1,500 individuals. The other locations have fewer than 500 attendees per conference so far.
PSQT (Practical Software Quality and Testing) conferences
PSQT conferences are held in the northern United States (usually June) and southern United States (usually October). Most conferences have at least one model-based improvement track.
ICSP (International Conference on Software Process)
ICSP is sponsored by the same folks as PSQT and focuses on both theoretical and practical aspects of process improvement.
Software Guru conference
This is a new conference, holding its first event in Mexico City in September 2006. It is primarily a Spanish-language conference, sponsored by the Mexican software-quality association AMCIS.
ISQC (International Software Quality Conference)
ISQC is sponsored by the American Society for Quality. Although it frequently has a process improvement track, it focuses more on software quality and testing practitioners.
PSM (Practical System and Software Measurement)
The annual PSM conference, usually in July, is a forum for the measurement aspects of process improvement and project management. Check its Web site (www.psmsc.com) for the latest information.
SE2 (South East Software Engineering conference)
SE2 is a small conference held in Huntsville, Alabama every year at the beginning of April. Its primary host is the Software Engineering Division of the U.S. Army Research, Development, and Engineering Command, and although there will be some distinctly DoD-focused presentations, there are often some really good sessions on process improvement–related topics. It’s a smaller conference, which can be nice if you want to have more interaction with people.
SSTC (Systems and Software Technology Conference)
SSTC is hosted annually by the Software Technology Support Center at Hill Air Force Base. Although it has been traditionally held in Salt Lake City, usually at the end of April or beginning of May, the U.S. Department of Defense is now considering moving the venue annually. This conference has a strong DoD focus, but there are usually a number of good process improvement-related presentations. Check for the latest information at www.stsc.hill.af.mil.
CMMI Technology Conference and Users Group
This conference is co-sponsored by the National Defense Industrial Association (NDIA) and the SEI. It is the premier conference for CMMI-specific research, development, and practice. It is usually held in Denver in the fall. Check the SEI Web site for more information.
AYE (Amplifying Your Effectiveness) conference/workshops
The AYE conference, usually held in November in Phoenix, Arizona, is a conference centered on humanistic change management. It’s a nice change of format from the usual 45-minute presentation. Each session is a half-day tutorial, so you get a chance to actually do something, and the sessions are generally very participative. Many of the sessions focus on various aspects of techniques based on Virginia Satir’s work, as translated into organizational settings by Jerry Weinberg and others.
NASAGA (North American Simulation and Gaming Association)
The NASAGA conference is usually held in the autumn somewhere in North America. The focus of this conference is participative, active techniques for supporting various organizational development and process improvement settings. Some of SuZ’s favorite techniques have come from this conference. If you have any interest in active learning, this is a great conference to attend and participate in. Most of the sessions focus on demonstrating a technique more than on presenting slides.
15.7.2 Web Sites of Interest
Here are the Web sites we mention in the text, as well as some additional ones to surf for a wide variety of process improvement and CMMI information.
The CMMI Survival Guide Site
www.awprofessional.com/title/0321422775
This is our site where you can find additional information on this book.
The Software Engineering Institute site has a multitude of material, such as:
• CMMI models, specifications, and other aids to applying CMMI
• Hundreds of process-related technical reports, many of which are useful and some of which have been written by your favorite authors (us, of course!)
If you haven’t already discovered the Software Engineering Information Repository, now’s the time. It’s a free resource, managed by the SEI, that accepts submissions from sources worldwide on many topics related to software engineering and process improvement. It’s a very rich resource set and sometimes can be daunting to navigate, but it has resources that don’t show up in a lot of other places, so it’s usually worth looking through the site.
www.sei.cmu.edu/tip/publications/toolkit
This is the CMMI in Small Settings Pilot Toolkit Web site.
www.sei.cmu.edu/cmmi/appraisals
The home page for information on SCAMPI, the standard appraisal method for CMMI.
The CMMI Transition Aids BSCW site is a repository of CMMI adoption aids moderated by Mike Phillips, program manager for CMMI. Phillips publishes job aids and presentations that have been made available for community use by individuals and organizations throughout the CMMI community. This is a place to find cross-model mappings; information about forthcoming CMMI project products; and “CMMI Today,” a presentation on what’s going on with CMMI that Phillips gives to SPINs and conferences. To access, go to the referenced url, click on “public area” and then click on “CMMI Transition Aids.”
The SEI Partner Network Directory of process improvement consulting and training resources.
Here you can find information on the International Process Research Consortium and their work determining the future of process improvement.
This is a public-plus-members site that focuses on project management in information technology settings and usually has good articles. The members-only area has lots of good templates and examples related to project management.
http://groups.yahoo.com/group/cmmi_process_improvement
This site has a wealth of information and conversations related to CMMI and process improvement.
This is the Grove Web site for the Drexler-Sibbett Team Performance Model and other team-related information.
www.ieee.org and www.computer.org
These sites are the source of a great deal of engineering process information.
This site has information and training courses from the originators of the Information Mapping technique.
This site is a community resource for Six Sigma training, consulting, and articles.
This site includes the ISO Standards Store.
Home for the National Defense Industrial Association. They sponsor scads of conferences on defense-related subjects, but of primary interest to you is the CMMI Technology Conference and Users Group. The proceedings and presentations from this conference are available here.
The UK organization that manages ITIL maintains this site.
This is the site for one of the great achievements in process improvement. The Software Engineering Lab at NASA operated for 25 years as the first (and probably only) organization to approach process improvement based on focused empirical studies. You can find information about Goal-Question-Metric as well as other QIP-related methods.
15.7.3 Journals, Magazines, and E-Zines of Interest
SPIP
This journal provides practical as well as theory-based articles on implementing software process improvement. It frequently has themed issues that explore a particular topic in depth.
SW Guru (Spanish)
This Spanish-language magazine (published in Mexico) focuses on many aspects of software and system development, including software process improvement and software standards.
Software Development
A commercially focused software development magazine that follows current trends in software methodologies and approaches, including material on process improvement.
IEEE Software
Published every two months, this is one of the highest regarded publicly available magazines for state-of-the-practice and state-of-the-art topics in the software field. The magazine includes frequent articles on process improvement.
CrossTalk
This magazine is published by the Software Technology Support Center (the U.S. Air Force organization that sponsors the SSTC). Because much of the research in process has been supported by the defense industry, this is often a place to find emerging authors and ideas. Issues are generally organized around a common theme, and process improvement is included in nearly every issue. Subscription is free.
Software Quality Professional (SQP)
Published by the American Society for Quality’s software division, this magazine focuses on the issues faced by software quality professionals. It’s a great resource for PPQA implementation, and it also has occasional process improvement articles.
Sticky Minds
Begun as a resource for software test professionals, this e-zine (www. stickyminds.com) also publishes book reviews and articles on process improvement, as well as other software quality topics.