4 Identify Risk
We are slaves to whatever we do not understand.
—Vernon Howard
In October 1993, Deputy Assistant Secretary of the Air Force Lloyd K. Mosemann wrote in a memo: “It appears that many problems of large-scale software development and maintenance, such as cost overruns, schedule slips, and unmet performance objectives, are in fact the result of the difficulties in effectively managing extremely large and complex undertakings—particularly in the areas of planning and baselining, control, visibility, and risk management. In all likelihood, effective management techniques are not as widely understood or utilized as they could be.” Mosemann articulated the very reason that risk management is rarely performed: most people do not know where to start. We can begin by understanding how to identify risks, the first step in the risk management process.
In this chapter, I discuss the risk identification process, which defines the activities and methods used to discover risk. I explain the techniques and tools proved to communicate identified risk effectively.
This chapter answers the following questions:
What are the activities of the risk identification process?
How can you discover unknown risks?
Why is a structured risk statement valuable?
4.1 Define the Risk Identification Process
I define risk identification success criteria in terms of the process goals, the results that we expect to achieve by using the process. We describe the risk identification process from two perspectives. The external view specifies the process controls, inputs, outputs, and mechanisms. The internal view specifies the process activities that transform inputs to outputs using the mechanisms.
4.1.1 Process Goals
The risk identification process is sufficient when it satisfies these goals:
Encourage input of perceived risk from the team.
Identify risk while there is time to take action.
Uncover risk and sources of risk.
Capture risk in a readable format.
Communicate risk to those who can resolve it.
Prevent project surprises.
These goals can be used as a checklist to ensure the process quality.
4.1.2 Process Definition
The risk identification process definition is shown as an IDEF0 diagram in Figure 4.1. Integrated Computer-Aided Manufacturing Definition (IDEF0) is a function modeling standard established in 1981 for defining manufacturing processes [AirForce81]. IDEF0 is a standard process definition notation used to describe a reusable process component for predictable risk identification. The diagram describes the top-level process by its controls, inputs, outputs, and mechanisms. Control determines when and how the process is executed. Input is an item required for a process transformation that meets the process entry criteria. Output is a result of a process transformation that has been successfully reviewed using process exit criteria. Mechanism determines the methods the process uses.
Process Controls. Project resources, project requirements, and the risk management plan regulate the risk identification process.
Figure 4.1 Risk identification process definition. Identify Risk encapsulates the activities of the process that transform inputs to outputs. Controls (at the top) regulate the process. Inputs (on the left) enter the process. Outputs (on the right) exit the process. Mechanisms (at the bottom) support the process.
The project resources constrain the scope of risk identification in terms of cost, time, and staff. When cost is a constraint, you can use cheaper methods to identify risk. When time is lacking, you can use quicker methods. When staff are not available, you can invite fewer people to participate in risk identification. If you cut corners due to project resources, you risk compromising process effectiveness.
Contractual requirements and organizational standards influence the project requirements for when to perform risk identification. You can derive requirements for risk identification from organizational standards that require reporting risks at project reviews. Contractual requirements can directly specify requirements for risk assessment, as in the following U.S. Marine Corps statement of work: “The contractor shall perform a continuing assessment of the risks associated with project and contract cost, schedule, and achievement of DemVal phase and project technical requirements.”
The risk management plan specifies who has the responsibility and authority for risk management activities. It documents the project procedures that specify how to implement the process. (Chapter 15 describes how to develop an effective risk management plan.)
Process Inputs. Uncertainty, knowledge, concerns, and issues are inputs to the risk identification process.
Uncertainty is what we do not know. It is thus a part of our assumptions and our doubts. For example, requirements that contain the phrase to be determined (TBD) are carriers of risk. Similarly, estimates not based on past performance contain more uncertainty (i.e., higher risk) than estimates based on historical data.
Knowledge is what we do know. We must use our previous experience in engineering systems and knowledge of the current project to identify software risk.
Concerns are what we fear. They cause anxiety, uneasiness, or worry. They are the troubles that affect us, which often relate to risk.
An issue is a matter that is unresolved with respect to other people. It is an open item that we work together to resolve. Issues can be risks when there are trade-offs that make a decision difficult.
Process Outputs. A risk statement and its associated risk context are output from the risk identification process.
A risk statement is a concise declaration of risk in a standard notation: Issue • probability • consequence. The simplicity of the risk statement makes it easy to read. The risk statement decomposes into the topic and the two main attributes of risk. The value of the risk statement is that it promotes increased understanding in risk communication. Further, it can help to determine whether an issue is a risk. For example, if the probability is 100 percent, there is no risk to resolve. It is too late to prevent the consequence.
The risk context provides the collateral information surrounding the risk statement (events, conditions, constraints, assumptions, circumstances, contributing factors, and related issues).
Process Mechanisms. A risk checklist, risk assessment, risk management form, and risk database are mechanisms of the risk identification process. Mechanisms can be methods, techniques, tools, or other instruments that provide structure to the process activities.
A risk checklist contains typical risk areas that relate to the checklist topic. Risk checklists can organize risks by, for example, contract type, maturity level, life cycle model, development phase, organizational structure, project size, application domain, or technology. They help to identify risks in a given area completely. I describe several risk checklists in section 4.2.
Risk assessment is a rigorous method for risk identification that uses a structured risk checklist in an interview-style meeting. I describe a risk assessment method further in section 4.3.
A risk management form is a mechanism for addressing risk systematically through a fill-in-the-blank template. I provide a risk management form in section 4.4.
A risk database is a repository of identified risks and associated information that logs the risk by assigning the next available sequential number and maintains the history of all identified risks. I outline the data fields of a risk database in section 4.5.
4.1.3 Process Activities
The activities of the risk identification process are the tasks necessary to transform uncertainty into a risk statement:
1. Conduct a risk assessment.
2. Identify risk systematically.
3. Define the risk attributes.
4. Document identified risk.
5. Communicate identified risk.
I discuss the risk identification process activities as sequential steps, but they can be iterative or parallel in execution.
Conduct a Risk Assessment. The risk assessment identifies risk and evaluates risk based on established criteria (e.g., likelihood of occurrence, consequence, and time frame for action). It provides a baseline of assessed risks that are managed by the project and thus is recommended early in any project. Subsequent risk assessments are recommended at major milestones or as significant project changes occur in areas such as cost, schedule, scope, or staffing. A streamlined risk assessment method [Hall95] is described in section 4.3.
Identify Risk Systematically. There are many ways to identify risk; several structure input so that it is easier to understand. The methods prescribed in this chapter fall under one or more of the following categories:
Checklist. Everyone can identify risks by using a checklist as a reminder of the possible risk areas. Section 4.2 defines several checklists used in risk identification.
Interview. People can identify risks when asked in a group interview session. The value of this method is that the interviewer is not a stakeholder in risk identification and is thus impartial. Moreover, a synergy develops from the discussion.
Meeting. Periodic meetings, such as weekly staff meetings, monthly team meetings, and quarterly project reviews, are appropriate for dialogue of risk information. Reserve a place for discussion of risk on meeting agendas.
Review. People can identify risks through a review of plans, processes, and work products.
Routine input. A risk management form is a template for routine input of identified risk. Forms should be available at any time in both hard copy and soft copy formats. (Section 4.4 defines a template for a risk management form.)
Survey. Selected categories of people can identify risks quickly and without prior preparation using a survey method. An anonymous survey does not require an individual's name but should provide a job category to understand the perspective of the survey response.
Working group. Teams can identify risks by thinking (e.g., brainstorming and meditation) and doing (e.g., modeling and simulation).
Define the Risk Attributes. After an issue is identified, you can be certain it is a risk by defining the major risk attributes of probability and consequence. A simple way to describe the likelihood of risk occurrence is by a subjective phrase. As shown in Figure 4.2, you can map the perceived likelihood to a quantitative probability. To define probability with a margin for error, you can use a range (e.g., 50 to 70 percent). When you have a low confidence in your perceived likelihood, it is best to use a rating system from 1 to 5, or high/moderate/low. Define the consequence in terms of an unsatisfactory outcome that would occur if the risk were realized. We often think of risk consequence as the cost, schedule, and technical effects. Objectives must be clearly stated and prioritized to develop accurate risk consequences. For example, the software design objectives can include flexibility, interoperability, maintainability, portability, reliability, reusability, and testability. Using these objectives, you can define specific technical consequences for a given design risk.
Figure 4.2 Perceived probability. A subjective phrase can be used to describe the likelihood of risk occurrence. The perceived probability is then mapped to a quantitative scale, where the probability is a number between 0 and 100 percent. Source: [DSMC83]
Document Identified Risk. The most concise way to specify a risk is to write a statement of risk containing a brief description of the risk issue, the probability, and the consequence in subjective terms. Using a standard format increases the readability of a risk, which makes it appear familiar. When we are comfortable writing risk statements, we have a known frame of reference for describing risk. When we use known techniques such as the risk statement, we feel more confident in our ability to manage the unknown aspects of risk. The value of a structured risk statement is its ability to simplify risk communication. The risk statement captures the essence of the risk. Document identified risk by writing a risk statement and elaborating the associated risk context. The corresponding risk context contains the what, when, where, how, and why of the risk issue.
Communicate Identified Risk. Communication of identified risk is best when it is both verbal and written. Verbal communication provides a dialogue for clarifying understanding: written communication provides a document for historical purposes. The advantage of verbal risk communication is that it can be heard, in person or over the telephone, and the listener has a chance to clarify the risk communication by asking questions. Face-to-face verbal communication is best the first time that significant risks are identified. A conversation over the telephone provides a personal touch when timeliness is necessary and distance prohibits an appearance in person. The advantage of written risk communication is that it can be read at a fixed location or distributed to a team at different sites. Bulletin boards and status reports are appropriate for a colocated team. E-mail and newsletters are appropriate for a distributed team.
4.2 Develop Risk Checklists
Risk checklists are easy to generate and provide a systematic way to identify risk. You can discover unknown risks that exist on your project by reviewing the project’s critical success factors, listing all items on the critical path of your schedule, and itemizing the project interfaces, internal and external. When I asked attendees at my Software Risk Management seminar to create a list of known risks by software development phase, four teams completed checklists for the requirements, design, code, and test phases in 15 minutes. You can avoid this work by using the existing SEI software risk taxonomy1 or your project work breakdown structure as a checklist.
1 Special permission to reproduce excerpts from Taxonomy Based Risk Identification, CMU/SEI–93–TR–6, © 1993 by Carnegie Mellon University, is granted by the Software Engineering Institute.
4.2.1 Software Risk Taxonomy
The SEI risk taxonomy is a structured checklist that organizes known software development risks from general classes to specific element attributes. The taxonomy provides a framework for identifying technical and programmatic software development risks [Carr93]. The SEI taxonomy2 is shown in Table 4.1 The taxonomy is organized into three major classes, which are further divided into elements. Each element is characterized by its attributes.
2 The SEI risk taxonomy is used in several SEI methods: Software Risk Evaluation [Sisti94], Team Risk Management [Dorofee94], and Continuous Risk Management [Dorofee96].
4.2.2 Work Breakdown Structure
The work breakdown structure (WBS) provides a framework for identifying specific project risk. A portion of a project WBS is shown in Figure 4.3. Working on activities that are not on the project WBS can be an indication of unknown risk. Any time you perform work not budgeted and scheduled, you detract from planned work. In this case, working on unplanned activities can cause problems, such as schedule slips and cost overruns, in other areas. Unplanned activities, or activities not decomposed properly, will steal time from planned activities. Only activities on the project WBS are visible at the cost accounting level.
4.3 Define the Risk Assessment Method
The risk assessment described in this section is an interview-style method to identify and appraise risks. The risk assessment objectives include training techniques for risk identification that are used throughout the project and providing a baseline of assessed risks for continued risk management. The method is based on concepts developed by the SEI Risk Program [Carr93]. Through collaborative efforts with SEI, a streamlined method of risk assessment was developed [Hall95].
Figure 4.3 Project work breakdown structure. The work breakdown structure can be used as a checklist of risk areas. Known risks are found in the activities listed; unknown risks are found in activities not listed.
4.3.1 Assessment Preparation
The risk assessment is performed using an assessment team trained in basic risk concepts, the risk assessment method, and the risk management process. The assessment team has a designated leader to coordinate activities with the project. The team reviews the project profile, which contains information on topics listed in Table 4.2, to understand the unique aspects of the project. The project profile should present the technical aspects and the current status of the project, including the organization chart and schedule. Existing charts should be used as much as possible. The project profile is usually briefed by the project manager or chief software engineer during risk assessment preparation. The individuals from the project selected to participate are informed about the risk assessment purpose and scheduled activities. The assessment preparation comprises the following tasks.
1. Select and train the assessment team.
2. Review the project profile.
3. Select the interview participants.
4. Prepare the risk assessment schedule.
5. Coordinate the meeting logistics.
6. Brief the project participants.
Table 4.1 Sei Software Risk Taxonomy
Table 4.2 Project Profile Contents
4.3.2 Interview Session
The assessment team conducts interviews of several peer groups using the SEI taxonomy-based questionnaire (TBQ), a risk identification mechanism that will be used throughout the project, and an interview protocol to identify and evaluate risks. The interview protocol creates a nonjudgmental and nonattributive environment so that tentative or controversial views may be voiced in the interview session [Carr93].
Figure 4.4 Risk assessment interview script. The interview script should be used as aguide to structure the interview session.
During the risk assessment, the assessment team rotates the roles of interviewer, recorder, and observer. The interviewer conducts the group discussion using an interview protocol in a question-and-answer format. The project participants identify risks in response to questions posed by the interviewer. The recorder documents risk statements and the results of the preliminary risk analysis, and the observer serves as timekeeper and notes details of the discussion, as well as group dynamics and overall process effectiveness.
The interviewer conducts the interview session in accordance with the interview script shown in Figure 4.4. The script ensures consistency among interview sessions and ensures that the interviewer remembers the five major components of the interview session. The interviewer asks the risk assessment team members to introduce themselves prior to conducting the interview session and reminds interviewees that no remarks will be attributed to an individual or interview group to emphasize confidentiality.
The interview session comprises the following tasks:
1. Interview the peer group.
2. Record the risks.
3. Clarify the risk statements.
4. Observe and document the process results.
4.3.3 Preliminary Risk Analysis
The participants assess the risk statements according to defined evaluation criteria. The assessment team evaluates the interview results after the participants have left the meeting room by reviewing the risk statements one at a time and classifying them according to the SEI risk taxonomy. Risks may be entered into the risk database as soon as the information is available.
The preliminary risk analysis comprises the following tasks:
1. Evaluate the identified risks.
2. Evaluate the interview session.
3. Categorize the risks.
4. Enter the risks in the risk database.
4.3.4 Results and Retrospective
Develop viewgraphs to summarize the findings of the interview sessions. Debrief the project manager and ask this person to speak to the project team regarding risk management after the assessment results are presented. Then present the risk assessment process and results to the entire project team, leaving time to respond to questions from the team. The project manager uses the risk database as a baseline of risks or merges it into an existing database. Feedback is documented on the risk assessment evaluation forms that are gathered from the interview participants after the results briefing. The project uses the baselined risks to continue the risk management process throughout the project. Risk management forms are available as the mechanism the project team will use to communicate new risks.
The results and retrospective comprise the following tasks:
1. Prepare the results briefing.
2. Debrief the project manager.
3. Brief the project team.
4. Evaluate the risk assessment.
5. Distribute the risk management form.
4.4 Develop the Risk Management Form
A risk management form is used as a risk identification mechanism that anyone may submit at any time to identify risks. The originator enters his or her name, the date, and a brief description of the risk. Figure 4.5 shows a template of a risk management form. A sample risk management form for an active risk is shown in Chapter 15.
4.5 Establish the Risk Database Schema
The risk database contains data fields to describe a risk completely. The database design should include links to requirements and pointers to related risks. The risk database schema is the design of the fields for the risk database. As a minimum, the risk database should contain the following data fields:
Log number
Date
Status
Originator
Risk category
Risk title
Probability
Consequence
Time frame
Project
Phase
Function
WBS element
Risk statement
Risk context
Risk analysis
Current priority
Previous priority
Risk resolution strategy
Risk action plan
Quantitative target
Indicator
Threshold
Trigger
Cost
Savings
Figure 4.5 Risk management form.
4.6 Summary
In this chapter, I described the process definition steps for identifying risk:
1. Define the risk identification process.
2. Develop risk checklists.
3. Define the risk assessment method.
4. Develop the risk management form.
5. Establish the risk database schema.
I described the risk identification process, which defines the activities and methods to uncover known and unknown risk. The activities of the risk identification process are as follows:
1. Conduct a risk assessment.
2. Identify risk systematically.
3. Define the risk attributes.
4. Document identified risk.
5. Communicate identified risk.
I identified sources of unknown risks that are inherent in software development. You can discover these risks by reviewing the following project components:
Work breakdown structure.
Critical success factors.
Critical path activities.
External interfaces.
I described techniques and tools proved to communicate identified risk effectively. The value of a structured risk statement is its ability to define a specific risk concisely, increase readability using a standard format, and simplify risk communication.
4.7 Questions for Discussion
1. Describe the risk identification process goals. Why is each goal important? Define quantitative success criteria for each goal.
2. In what way do project resources, project requirements, and the risk management plan regulate the risk identification process? Is there a relationship among these process controls? Discuss why there is or why there is not.
3. What is a risk assessment? Cite three reasons to perform a risk assessment early in the project. Imagine that you were brought in to replace a retiring project manager of a project in the design phase. Would you delegate the task of performing a risk assessment? Discuss the ways a baseline of assessed risks would be valuable to you.
4. List five methods to identify risk. Describe how you would perform a risk assessment using each of these methods. Compare and contrast the methods in terms of their efficiency and effectiveness. For example, if it is more effective to involve more people, then which methods enable more people to participate?
5. List three reasons to categorize risk. Do you think the number of risks identified in a particular category is a significant measure? Discuss why you do or do not think so.
6. Write a risk statement. What are five benefits of using a structured format for documenting risk? How do you think communication of identified risk changes with the use of a risk statement? Explain your answer.
7. You just found out that your software subcontractor has filed for bankruptcy protection. How will you communicate this significant risk to your management? Do you plan to communicate the risk as soon as possible, or take the time to assess it? What information will you communicate to your management?
8. Develop a risk checklist for the requirements, design, code, or test phase of software development. Which phase do you think has the greatest risk? Explain your answer.
9. Discuss the concept of confidentiality in risk assessment. When is confidentiality necessary? In your opinion, what are the advantages and disadvantages of identifying risks in peer groups?
10. How does knowing your risks provide opportunities to manage and improve your chances of success? Explain your answer.
4.8 References
[AirForce81] Integrated Computer Aided Manufacturing Architecture, Part II, Vol. IV: Function Modeling Manual (IDEF0). AFWAL-TR-81-4023. Wright-Patterson Air Force Base, OH: Air Force Systems Command, June 1981.
[Boehm89] Boehm B. IEEE Tutorial on Software Risk Management. New York: IEEE Computer Society Press, 1989.
[Carr93] Carr M, Konda S, Monarch I, Ulrich F, Walker C. Taxonomy based risk identification. Technical report CMU/SEI-93-TR-6. Pittsburgh, PA: Software Engineering Institute, Carnegie Mellon University, 1993.
[Dorofee96] Dorofee A, et. al. Continuous Risk Management Guidebook. Pittsburgh, PA: Software Engineering Institute, Carnegie Mellon University, 1996.
[Dorofee94] Dorofee A, et. al. Introduction to team risk management. Special technical report CMU/SEI-94-SR-001. Pittsburgh, PA: Software Engineering Institute, Carnegie Mellon University, 1994.
[DSMC83] Defense Systems Management College. Risk Assessment Techniques. Fort Belvoir, VA, 1983.
[Hall95] Hall E, Natwick G, Ulrich F, Engle C. Streamlining the risk assessment process. Proc. Seventh Software Technology Conference, Salt Lake City, UT, April 1995.
[Sisti94] Sisti F, Joseph S. Software Risk Evaluation Method. Version 1.0. Technical report CMU/SEI-94-TR-19. Pittsburgh, PA: Software Engineering Institute, Carnegie Mellon University, 1994.