Chapter 11
Risk Management
After completing this chapter, you will be able to:
- – understand risk management;
- – know the main standards and models that include requirements for risk management;
- – understand the risks that can affect the quality of a software;
- – understand the techniques used to identify, prioritize, document, and mitigate risks;
- – understand the roles of participants in risk management;
- – understand the human factors involved in risk management;
- – understand how to conduct risk management for very small entities;
- – recognize the requirements for risk management in a software quality assurance plan.
11.1 Introduction
Software engineers and project managers are eternal optimists. When planning a project, they often assume that everything will go as planned. Reality is very different as every software project includes risks. Risk management is recognized as a proven practice in the software industry. According to Charrette (1992) [CHA 99], many software professionals have the wrong perception of risk management. They see it as a necessary but uninteresting task to be done before the really interesting coding work begins. It is perceived as over management or as another bureaucratic activity that prevents the organization from achieving its objectives.
In some organizational cultures, those that raise the flag to indicate a new risk are often perceived as negative or as trouble makers. Management will often react by attacking these individuals instead of attacking the risks highlighted. These organizations are often in reactive mode. When a risk becomes a real problem, they try to mitigate it and then manage it by adding personnel to a project that is already late, for example. When these strategies fail, the organization goes into crisis management. Now it needs to put out fires.
There is a large number of sources of risk that are both external and internal to a software project. Figure 11.1 illustrates some of these sources.
Figure 11.1 Some software project internal and external risk sources.
Source: Adapted from Shepehrd (1997) [SHE 97].
With the growing complexity of software and the demand for even better, bigger, and more performing software, the industry is now becoming a high risk endeavor. When software development project teams do not manage their risks, they become vulnerable to major rework, additional costs, late delivery, or simply project failure.
Figure 11.2 illustrates the context within which software is developed. Risk management, as presented in this chapter, covers project management, that is, the risks related to the developed processes and products, although the organizational context can also present risks.
Figure 11.2 Software projects—many surrounding contexts [CHA 99].
At the beginning of a software project, there are things that you know, things that you know you do not know (but that you need to understand) and unknown–unknowns, which are things that you do not know at all. The ones that you should worry about are the unknown–unknowns. These are the items that come as surprises and are unpredictable. Risk management as presented in this chapter, aims at managing the first two types of unknowns listed above. With respect to the unknown–unknowns, you will become quicker at identifying them with experience and through the use of risk management over a number of years.
The following text box describes some of the definitions of risk.
For the PMI, the risk management objectives of a project are: to augment the probability and the impact of positive events as well as reduce the probability and impact of negative events in the project [PMI 13]. In this chapter, we use a more widely accepted definition of risk, which is more pessimistic.
Risk management allows for raising the awareness of a doubt before it becomes a crisis. This technique improves the chances of successfully completing the project and reduces the impact of risks that cannot be avoided. Effective risk analysis and its management, for a given project, helps in identifying the hypothesis, constraints, and objectives that may change for the worse.
Boehm states four good reasons to use software risk management techniques (adapted from Boehm (1989) [BOE 89]):
- – avoid catastrophic events in software projects, including budget and schedule overruns, defective software products and production failure;
- – avoid rework caused by requirements, by design or by source code that is wrong, missing or ambiguous which generally accounts for 40–50% of the total cost of software development;
- – avoid overkill by using detection and prevention techniques when the risk is minimal or non-existent;
- – encourage a win–win software solution where the customer gets the product he needs and the supplier receives the expected benefits.
Figure 11.3 illustrates the typical progress of a project. We see that, in the early analysis phase, a very small part of the budget is spent. On the other hand, a large proportion of the budget is already committed before this phase is even completed. This situation raises the risk profile of the project as there may be budget overruns.
Figure 11.3 Typical expense curve of a project [FOR 05].
We cannot only be involved with low risk projects. A risk represents a competitive advantage or a deciding factor. Since every software project is unique, there is no miracle recipe for a development project where some, often unpleasant, surprises occur. By definition, a software development project always includes some risks. The saying “forewarned is forearmed” applies perfectly to risk management. It is better to be proactive than reactive with respect to software development.
Risk management is only one of the elements of the project decision process in an organization. Risk can appear at software, system, or organizational levels. These three risk management levels are intimately linked. To fully understand these links, it is necessary to identify what is valued in the organization or the project. For example, is innovation and risk taking well regarded or frowned upon? The organizational culture will impact the tolerance for risk in software projects.
The second issue to clarify and take into account is the risk management process itself. It needs to be documented, known, reproducible, measurable, deployed, and used.
Behaviors should also be considered. For example, is the organization's communication about risks open and honest? When there is a risky situation in the project, we need to think about what we want the project team members to do or not to do. What would you like the individuals to do when they are faced with a risky situation or when faced with a situation they feel they may fail?
Interestingly, risk management may seem easy to do at first. In practice it can become a complex process, because risks are not tangible, only the resulting problems are. Risks are potential problems. When you try to lower their probability or consequences, the question is: will the investment in managing this risk improve the chance of success for the project?
Most project risks are a combination of political, social, economic, environmental, and technical factors. It can be difficult to isolate the factors and quantify the risk when they depend on individual perceptions. Given that risk is generally perceived, its estimation and probability is also generally perceived and can be biased. In organizations that have been managing risk for a long time, information accumulated from many software projects allow managers to anticipate and manage risk better with the use of quantitative measures.
Other difficulties may arise due to the varying levels of tolerance to risk from different stakeholders (e.g., marketing team, development team, or the client). For example, a lending agent in a bank and a professional sports player have different tolerance levels depending on the requirements of their profession. If they exchange jobs they will have to adapt their levels of tolerance to risk.
The three main risk factors in software development are (adapted from Charette (2006) [CHA 06]):
- – an unrealistic attitude: in the software industry, it is common to make promises and under estimate effort. Unrealistic objectives are especially observed in complex projects;
- – lack of discipline: we note, with regards to the CMMI, that the majority of software organizations are not disciplined or use deficient development practices, such as a bad project management approach that will destroy the project faster than any other risk factor, apart from unrealistic attitudes;
- – political games: some projects are not planned or executed in a completely objective or rational way. They are part of other, more organizational, political issues. Most software project managers have difficulty managing these situations, as well as understanding their influence on the success of the project.
Risk management is a good tool to continuously review the feasibility of project plans, identify and resolve problems that could impact the life cycle processes of the project, product quality, and performance with the objective of improving project management processes.
Software development risk management does not guarantee project success or the total absence of risks. It does not relieve stakeholders of their social, moral, financial, or legal obligations.
Figure 11.4 outlines the SWEBOK body of knowledge for software management. Among the major categories, risk management is highlighted in the left hand side of the diagram.
Figure 11.4 Risk management in the SWEBOK [SWE 14].
11.1.1 Risk, the Cost of Quality and Business Models
The importance of software business models and the cost of quality have already been discussed. In regards to the cost of quality, risk is considered as an element of prevention costs, that is, the costs incurred by an organization to prevent defects in different phases of its software life cycle. With respect to risk management, prevention costs are identification costs, along with the analysis and execution of risk mitigation measures. Table 11.1 describes the different prevention elements.
Table 11.1 Risk Management Prevention Costs
| Major category | Sub categories | Definition | Typical elementary costs |
| Prevention costs | Establish quality fundamentals | Efforts to define quality, set quality objectives, standards and checklists. Analysis of compromises linked to quality. | Definition of success criteria, acceptance tests and quality standards. |
| Interventions oriented toward projects and processes | Efforts dedicated to prevent bad quality and improve the quality of the process/project. | Training for process improvement, measurement and analysis. Risk identification, analysis, and mitigation. |
Source: Adapted from Krasner (1998) [KRA 98].
Risks are at the foundation of all the business models. Software development practices should be chosen in response to the inherent risks of each model. To minimize losses and errors, developers must make a careful selection of software quality assurance (SQA), verification and validation, and risk management practices.
11.1.2 Costs and Benefits of Risk Management
The costs and benefits of risk management can vary greatly: from not managing risks at all to controlling all the project risks. The objective is to strike a balance where risk is minimized at an optimum cost.
All new approaches, like implementing risk management for the first time, will require an initial investment to document the activities of this process, training personnel, and ramping up its use in all projects. The most important investments are made when projects apply the process for mitigating the probability and consequences of risk, for example, with the development of a prototype to better understand the customer requirements or with the additional hiring of an external consultant to conduct a feasibility study.
Risk management offers a structured mechanism to give better visibility to threats perceived by the project team. It also allows for the quantifying of the schedule slippage if some risks materialize. Project performance becomes more predictable and project reviews are more effective. It helps the project manager to plan a contingency budget (e.g., in money and time) to avoid errors made in the past (e.g., overconfidence). Risk management activities should start with the request for information, before an acquisition project is initiated. This technique can also be used to assess the capability of a supplier to deliver a critical component, on time and at the quality level specified.
The advantages associated with the effective use of risk management in projects are [ISO 17 and SEI 10a]:
- – a defined and executed risk management strategy;
- – potential problems, that is, risks that could influence the success of the project are identified;
- – the probability of risk and their consequences are understood;
- – risks are ranked by priority highlighting those that will be tracked closely;
- – appropriate mitigation solutions are developed proactively, taking into account the project context, diminishing crisis situations where risks become problems;
- – mitigation solutions are chosen for risks that have surpassed threshold limits;
- – project risk information is captured, analyzed, and exploited with the objective of improving the risk management procedures and policies.
11.2 Risk Management According to Standards and Models
This section briefly presents the standards and models that describe risk management requirements. First, we present the requirements of ISO 9001. Then we discuss the ISO/IEC/IEEE 12207 that describes all the life cycle processes including the recommended risk management process. Next, a section is dedicated to the ISO/IEC/IEEE 16085 standard [ISO 06a]. Risk management is also covered by the CMMI. Given that software development is almost always associated with a project, the point of view of the PMBOK® Guide of the Project Management Institute (PMI) is presented. The following section presents a discussion on how to apply risk management to very small entities using the ISO 29110. Finally, the requirements for risk management included in a SQA plan are described.
11.2.1 Risk Management According to ISO 9001
It is important to point out that ISO 9001 uses a risk-based thinking approach, among others [ISO 15]. A risk-based approach allows the organization to identify factors that may create a gap between its processes and its quality management system (QMS) and the expected results. It also allows for the implementation of a preventive process to limit any negative effects and to capitalize on improvement opportunities.
Clause 6.1 describes the actions to address risks and opportunities, the quality objectives of the QMS and their achievement, and modifications to the QMS. It has a number of requirements that are listed in the next text box.
11.2.2 Risk Management According to ISO/IEC/IEEE 12207
The purpose of the risk management process, according to the ISO 12207 [ISO 17] standard, is to identify, analyze, treat, and monitor the risks continually. The risk management process should be a continuous process for systematically addressing risk throughout the life cycle of a system, software product, or service. It can be applied to risks related to the acquisition, development, maintenance, or operation of a system.
As a result of the successful implementation of the risk management process [ISO 17]:
- – risks are identified;
- – risks are analyzed;
- – risk treatment options are identified, prioritized, and selected;
- – appropriate treatment is implemented;
- – risks are evaluated to assess changes in status and progress in treatment.
11.2.2.1 Activities and Tasks of the Risk Management Process
In accordance with the policies and procedures of the organization concerning risk management, the project shall implement the following activities [ISO 17]:
- – plan risk management;
- – manage the risk profile;
- – analyze risk;
- – treat risk;
- – monitor risk.
11.2.3 Risk Management According to ISO/IEC/IEEE 16085
Risk management according to the ISO 16085 [ISO 06a] standard supports the acquisition, supply, development, operation, and maintenance of products and services by providing a series of process requirements that can address a wide variety of risks. The purpose of this standard is to provide suppliers, acquirers, developers, and managers with a single set of process requirements suitable for the management of a broad variety of risks [ISO 06a].
This standard does not describe risk management techniques. It defines a risk management process that can be used with many different techniques. The use of this standard does not require a specific life cycle process. The measurement process, described within ISO 15939 [ISO 17c] and described in an earlier chapter, works closely with the risk management activities described in the ISO 16085 standard to both identify and quantify risks.
The ISO 16085 risk management process, as illustrated in Figure 11.5, is continuously executed during all the activities of the life cycle of the product. It is recommended that this process include the following activities [ISO 06a]:
- – plan and implement risk management;
- – manage the project risk profile;
- – perform risk analysis;
- – perform risk monitoring;
- – perform risk treatment;
- – evaluate the risk management process.
Figure 11.5 ISO 16085 recommended risk management process [ISO 06a].
Source: Standards Council of Canada.
The risk management process is initiated using the information requested by the stakeholders of the technical and management process of an organization (see rectangle number 1 of Figure 11.5) to make decisions that include risks. The risk management process activities are [ISO 06a]:
- – during the execution of the activity entitled “Plan and implement risk management” (rectangle number 2 of Figure 11.5), the policies regarding the general guidelines under which risk management will be conducted, the procedure to be used, the specific techniques to be applied, and other matters relevant to risk planning are defined. The risk management plan (RMP) can include the following topics:
- overview;
- scope;
- reference documents;
- glossary;
- risk management overview: describe the specifics of risk management for this project or organization's situation;
- risk management policies: describe the guidelines by which risk management will be conducted;
- risk management process overview;
- risk management responsibilities: define the parties responsible for performing risk management;
- risk management organization: describe the function or organization assigned responsibility for risk management within the organizational unit;
- risk management orientation and training;
- risk management costs and schedules;
- risk management process description;
- risk management process evaluation;
- risk communication: describe how risk management information will be coordinated and communicated among stakeholders and interested parties (i.e., those who are interested in the performance or success of the project or product but not necessarily of the organization) such as what risks need reporting to which management level;
- RMP change procedures and history.
- – during the execution of the activity entitled “Manage the project risk profile” (rectangle number 3 of Figure 11.5), the context of current and historical risk management as well as the risks states are documented. The risk profile of the overall project is the sum of all the individual risk profiles;
- – information on the project risk profile is constantly updated by the “Perform risk analysis” activity (rectangle number 4 of Figure 11.5) that identifies risks, determines their probability, lists their consequences, assesses the risk of exposure and prepares the risk action requests for risks that cross their established thresholds;
- – the risk mitigation recommendations, the state of other risks, and their mitigation proposals are sent to management to be reviewed (rectangle number 5 of Figure 11.5). Management decides whether to approve that a mitigation of critical risks should be performed. The mitigation plans are then developed. These plans will be included in the project management activities to be coordinated with the other project plans and current activities;
- – all the risks are then monitored until they stop posing a threat. For example, they are removed during the activity entitled “Perform risk monitoring” (rectangle number 6 of Figure 11.5). New potential risks are then investigated;
- – a periodic assessment of the risk management process is necessary to ensure its effectiveness. During the activity entitled “Evaluate the risk management process” (rectangle number 7 of Figure 11.5), information, such as feedback, is documented to improve the process or the organizational or project's capacity to better manage risks. The improvements identified following a risk assessment are then used by the process entitled “Plan and implement risk management” (rectangle number 2 of Figure 11.5).
11.2.4 Risk Management According to the CMMI Model
The CMMI®-DEV contains many process areas that discuss risks. As illustrated in Figure 4.9, in the staged representation of the CMMI® model, risks are discussed in two separate process areas of maturity level 2 [SEI 10a]:
- – project planning: one of the project planning practices, SP 2.2, is listed as identifying and analyzing project risks. Four sub-practices are:
- identify risks;
- document risks;
- review and obtain agreement with relevant stakeholders on the completeness and correctness of documented risks;
- revise risks as appropriate.
The typical outputs of these practices are:
- identified risks;
- risk impacts and probability of occurrence;
- risk priorities.
The CMMI model also proposes examples of risk identification and analysis tools such as: risk taxonomy for determining the source and categories of risks, checklists, and brainstorming sessions;
- – project monitoring and control: an appreciation of the project progress is obtained allowing corrective actions to be taken when the project performance diverges significantly from its plan. One of the specific practices, SP 1.3, discusses the need to monitor the identified risks. Three sub-practices state that:
- periodically review the documentation of risks in the context of the project's current status and circumstances;
- revise the documentation of risks as additional information becomes available;
- communicate the risk status to relevant stakeholders.
An example work product of this SP is the records of project risk monitoring.
At maturity level 3, the “risk management” process area focuses on the prevention of potential problems before they appear. The purpose of the risk management process area is to identify potential problems before they occur so that risk handling activities can be planned and invoked as needed across the life of the product or project to mitigate adverse impacts on achieving objectives. This process area includes the following specific objectives and practices [SEI 10a]:
- – SG 1 Prepare for risk management
- SP 1.1 Determine risk sources and categories
- SP 1.2 Define risk parameters
- SP 1.3 Establish a risk management strategy
- – SG 2 Identify and analyze risks
- SP 2.1 Identify risks
- SP 2.1 Evaluate, categorize, and prioritize risks
- – SG 3 Mitigate risks
- SP 3.1 Develop risk mitigation plans
- SP 3.2 Implement risk mitigation plans
We can see that at maturity level 2, two of the process areas, that is, project planning and project monitoring and control, aim at risk identification and mitigation when they appear, whereas at maturity level 3, the risk management process proposes practices to ensure a systematic and continuous predictive practice for the planning, anticipation, and analysis of risk.
CMMI also addresses agile topics [SEI 10a]: some risk activities are already part of agile methodologies, for example, some technical risks can be addressed by early experimentation and experimenting early failures or by executing a spike outside the scope of the current iteration. However, the risk management process suggests a more systematic approach of technical management of risks. Such an approach can be included in agile iterations and meetings, as well as in iteration planning and task assignments.
11.2.5 Risk Management According to PMBOK® Guide
The Project Management Body of Knowledge (PMBOK® Guide) of the Project Management Institute [PMI 13] includes nine knowledge areas and the management of project risk is one of them.
Project risk, according to the PMBOK® Guide, is an uncertain event or condition that, if it occurs, has a positive or negative effect on one or more project objectives, such as schedule, costs, content, or quality (where the schedule objective is to deliver the product within the agreed upon delay and the cost objective is to deliver the product within the agreed upon budget, etc.) [PMI 13].
Figure 11.6, of the PMBOK® Guide, describes the difference between the influence of the stakeholders’ risks and uncertainties and the costs of modifications as the project progresses.
Figure 11.6 Impact of variables as the project progresses [PMI 13].
The PMBOK® Guide, proposes that risk management includes six processes [PMI 13]:
- Plan risk management
- – The process of defining how to conduct risk management activities for a project.
- Identify risks
- – The process of determining which risks may affect the project and documenting their characteristics.
- Perform qualitative risk analysis
- – The process of prioritizing risks for further analysis or action by assessing and combining their probability of occurrence and impact.
- Perform quantitative risk analysis
- – The process of numerically analyzing the effect of identified risks on overall project objectives.
- Plan risk responses
- – The process of developing options and actions to enhance opportunities and to reduce threats to project objectives.
- Control risks
- – The process of implementing risk response plans, tracking identified risks, monitoring residual risks, identifying new risks, and evaluating risk process effectiveness throughout the project.
11.2.6 Risk Management According to ISO 29110
Very Small Entities (VSEs) manage software project risks “in the small.” For example, the projects are executed on short schedules and we may not have had the time to think about risks and the ways to mitigate them. It is therefore necessary to be alert because we need to react very quickly when a risk emerges and rapidly becomes a problem because the schedule of the project is short. In these VSEs, the development teams are very small and when a problem occurs there is often no one available to address it.
Additionally, the VSE may already be in crisis mode and team members may not necessarily have the expertise or authority to address a risk. In VSEs, the authority often resides with the same individual that solves all the problems.
The authors of the VSE standard have included some risk management tasks to help. The following text describes risk management as presented in the Basic profile. The Basic profile refers to VSE who develop one project at a time with only one development team. The Intermediate and Advanced Profiles for VSEs require more involved risk management processes as they develop more than one project at a time with several teams.
The Basic profile of ISO 29110 [ISO 11e] has selected some of the expected outcomes from the ISO 12207 [ISO 17]. One objective of the project management process is “Risks are identified as they develop and during the conduct of the project.” With these objectives in mind, tasks, roles, inputs, and outputs were described in an ISO 29110 management and engineering guide [ISO 11e].
Table 11.2 describes the risk management tasks suggested during the project planning activity of the project. Roles in these activities are: the project manager (PM), the technical lead (TL), and the work team (WT).
Table 11.2 Risk Management Task During Project Planning [ISO 11e]
| Role | Task list | Input work product | Output work product |
PM TL |
PM.1.9 Identify and document the risks which may affect the project |
All elements previously defined | Project Plan
|
Table 11.3 describes the risk management tasks to be done during the project implementation.
Table 11.3 Risk Management Task List During Project Implementation [ISO 11e]
| Role | Task list | Input work products | Output work product |
PM TL WT |
PM.2.3 Conduct revision meetings with the work team, identify problems, review risk status, record agreements, and track them to closure |
Project Plan Progress Status Record Correction Register Meeting Record |
Meeting Record [updated] |
Table 11.4 presents the three risk management tasks during the monitoring and control activity of the project (only the elements related to risk management are listed).
Table 11.4 Risk Management Tasks List During the Assessment and Control Activity [ISO 11e]
| Role | Task list | Input work products | Output work products |
PM TL WT |
PM.3.1 Evaluate project progress with respect to the Project Plan, comparing:
|
Project Plan Progress Status Record |
Progress Status Record [evaluated] |
PM TL WT |
PM.3.2 Establish actions to correct deviations or problems and identified risks concerning the accomplishment of the plan, as needed, document them in Correction Register and track them to closure. |
Progress Status Record [evaluated] | Correction Register |
PM TL WT |
PM.3.3 Identify changes to requirements and/or Project Plan to address major deviations, potential risks or problems concerning the accomplishment of the plan, document them in Change Request and track them to closure. |
Progress Status Record [evaluated] | Change Requests [initiated] |
Software work products related to these tasks have also been developed. Table 11.5 describes the proposed content of the project plan and of progress reports (only the elements related to risk management are listed).
Table 11.5 Proposed Content of the Project Plan and the Progress Status Records [ISO 11e]
| Name | Description | Source |
| Project Plan | Presents how the project processes and activities will be executed to assure the project's successful completion, and the quality of the deliverable products. It includes the following elements which may have characteristics as follows:
The applicable statuses are: verified, accepted, updated, and reviewed. |
Project Management |
| Progress Status Record | Records the status of the project against the project plan. It may have the following characteristics:
The applicable status is: evaluated |
Project Management |
11.2.7 Risk Management and the SQA According to IEEE 730
We have already discussed that SQA ensures that processes are established, managed, maintained, and applied by skilled and qualified staff and that the activities and tasks performed are commensurate with product risk. Software systems are increasingly developed to perform tasks that can cause harm to living things, physical structures, and the environment. A fundamental principle of this standard is to first understand software product risk and then to ensure that the planned SQA activities are appropriate for the product risk. This means that the breadth and depth of SQA activities defined in the SQA plan are determined by and derived from software product risk.
The risk management descriptions for a project can be documented separately, as part of the project plan or as a section of the SQA plan. What is important is that it be present, complete, and available for reviews and audit. Following is a list of issues that the project manager and the SQA should consider [IEE 14]:
- – prepare the SQA plan and identify the SQA activities and tasks for the project consistent with the software product risks established for the project;
- – the SQA function will analyze product risks, standards, and assumptions that could impact quality and identify specific SQA activities, tasks, and outcomes that could help determine whether those risks are effectively mitigated;
- – analyze the project and adapt the SQA activities to correspond to the risk;
- – identify and track project changes that require further SQA planning, including changes to requirements, resources, schedules, project scope, priorities, and product risk.
- – determine whether the defined software life cycle processes selected by the project team are appropriate, given the product risks.
For IEEE 730, software product risk refers to the inherent risks associated with use of the software product (e.g., safety risk, financial risk, security risk). Software product risk is differentiated from project management risk. Techniques for addressing software product risk are discussed in section 4.6.2 of this standard and in Annex J of this standard [IEE 14]:
- – are potential product risks known and well documented?
- – are potential product risks understood so that SQA activities can be planned in a manner appropriate to the product risk?
- – has the scope of product risk management to be performed been determined?
- – have appropriate product risk management strategies been defined and implemented?
- – will a software integrity/criticality level be established, if appropriate?
- – does the project team have adequate training in product risk management techniques?
- – is the project team planning to adjust their activities and tasks in a manner consistent with product risk?
- – are the breadth and depth of planned SQA activities proportional to product risk?
- – are risks identified and analyzed as they develop?
- – has the priority in which to apply resources to the treatment of these risks been determined?
- – are risk measures appropriately defined, applied, and assessed to determine changes in the status of risk and the progress of the treatment activities?
- – has appropriate treatment been taken to correct or avoid the impact of risk based on its priority, probability, and consequence or on the defined risk threshold?
- – based on product risk, do the project tools, especially those used for the product SQA, require validation before they can be used on this project?
- – have additional project risks appeared that could prevent SQA from accomplishing their project responsibilities?
Finally, we would like to note that high risk industries, such as medical devices, transportation, and nuclear energy have additional risk management recommendations that originate from national monitoring bodies. For example, for medical devices, risk management is not the same as the risk management defined in IEEE 730. Please refer to these additional guidelines when working within these high risk industries.
11.3 Practical Considerations for Risk Management
In this section, we discuss a practical risk management approach step by step. This risk approach has been adapted from Boehm (1991) [BOE 91]. To facilitate its implementation, we have added a few tools that are easy to use. As illustrated in Figure 11.7, risk management has two major steps: risk evaluation followed by risk control. We have added another activity entitled “Lessons learned” for analyzing risks once the project is completed in order to update the risk process (e.g., checklist) of the organization.
Figure 11.7 Risk management activities.
The risk evaluation step consists of three activities: risk identification, risk analysis, and risk prioritization. The risk control step consists of three activities: risk plan development, risk mitigation, and risk monitoring.
The following text describes each activity along with helpful tools for their implementation.
11.3.1 Risk Evaluation Step
The risk evaluation step consists of three activities: risk identification, risk analysis, and assigning priorities.
11.3.1.1 Risk Identification Activity
Risk identification first produces a list of potential risks that are specific to the project and are susceptible of compromising its success. To do this, the following tools and techniques can be used: documentation review, using the organizational risk checklist, brainstorming with the project team, conducting interviews, strengths and weaknesses, opportunities and threats analysis (SWOT), using past experience, project lessons learned reviews and cause and effect diagrams.
Experienced personnel are often available in an organization. These individuals are well placed to propose ideas to resolve project problems and identify other potential problems that the project team did not consider. Table 11.6 describes the most common software project risks that are reported according to Boehm.
Table 11.6 List of Most Common Risks
| Risk element | Risk management techniques |
| Personnel shortfall | Attract talented personnel (increase salaries), team training and cross functional training. |
| Schedules and budgets that are too optimistic | Estimates made using different techniques, incremental development, reuse, project hypothesis analysis (e.g., technology selected is adequate and available). |
| Incorrect functionalities and properties developed | User involvement, prototype, task analysis, user survey, user guide available early in the project. |
| Incorrect user interfaces developed | Prototyping, scenarios, task analysis, user participation. |
| Gold plating | Requirements scrubbing, prototype, cost-benefit analysis, fixed budget. |
| High requirements churn and requirements creep | Incremental development (to push forward changes to later iterations), stricter control from the configuration control board. |
| Defective components originating from outside the project | Benchmarking, inspection, reference verification, compatibility analysis. |
| Shortfalls occurring in the project tasks outside the project | Reference verification, audit, CMMI evaluation, award-fee contracts, increase the contract terms. |
| Real-time performance problems | Simulation, benchmarking, modeling, prototyping. |
| Capacities, both human and technical, pushed to their limits | Prototype, technical analysis, cost-benefit analysis, technology readiness level evaluation. |
Source: Adapted from Boehm (1991) [BOE 91].
A software project can face different types of risks: technical, management, financial, personnel, and other resources (adapted from Westfall (2010) [WES 10]):
- – technical risks include problems associated with project size, project functionality, platforms, methods, standards, or processes. These risks can originate from excessive constraints, inexperience, wrongly defined parameters or dependencies with organizations that are out of the control of the project team;
- – management risks include not enough planning, inexperience in management and training, communications problems, a lack of authority, and financial control problems;
- – contractual risks and judicial risks include changes to requirements, market influences, health and safety issues, government regulation, and product warranty;
- – personnel risks include late acquisition of personnel, inexperienced and untrained personnel, ethical and moral issues, personnel conflicts, and productivity issues;
- – other resources of risks originate from the unavailability or late delivery of equipment, tools, and environment configurations as well as slow response time.
NASA has developed a tool, called Technology Readiness Levels (TRLs), to help assess the risk involved when a project wants to include hardware or software technology that could pose major technological risks. TRLs have been developed to assess software risks. The following text box describes the TRLs.
Many techniques can help with risk identification such as interviews, brainstorming, decomposition, project assumption analysis, documentation about the unknowns of a project, critical path analysis, reviewing the risk list generated by end of project reviews, and using risk taxonomies and checklists. In addition, a proper work environment facilitates the communication of risks.
A risk statement typically includes two parts: the risk condition and its potential consequences. The condition is a statement of the potential problem that “describes the main circumstances, the situation generating doubt, anxiety or uncertainty” [DOR 96]. A consequence is a brief description explaining the potential loss or negative outcome if this condition appears during the project execution. For example, if the team does not deliver their components in compliance with the quality level expected by the customer, there will be a need to raise the effort required, with overtime, for the next three weeks.
A simple and easy tool to use for identifying risks is a checklist. The following list includes typical risks (see the following text box).
Once risks are identified, the next step is to document them. Table 11.7 provides an example of a risk documentation grid. The column on the left is entitled “Risk identification number” and is a number assigned to each risk by the project manager (e.g., using a simple sequence). The column entitled “Risk description” describes the risk by using the following formulation: “if event X happens then its consequence will be Y.” For example, “if the estimation of the effort is incorrect by 10%, then the product delivery could be two weeks late.”
Table 11.7 Risk Documentation Grid
| Risk identification number | Risk description | P | C | E | Risk mitigation |
| 1 | |||||
| 2 | |||||
| 3 |
Source: Adapted from Wiegers (1998) [WIE 98].
11.3.1.2 Risk Analysis Activity
Once the risks are defined and documented, we proceed with the analysis of each risk. The probability of each risk and the impact is identified as well as the possible interactions between risks. The tools and techniques for this activity are: cost models, quality factor analysis (e.g., reliability, availability, security), sensitivity analysis, and decision trees [BOE 91].
Here is a list of questions that can facilitate the analysis:
- – when could the event happen?
- under what circumstances?
- when should you act to avoid or lessen the consequences?
- what could happen afterward?
- – what is the probability of occurrence?
- – what is the consequence?
- – in what way can we quantify the consequence?
- – what can we control or influence?
- the probability of this event occurring?
- the probability of possible results?
- the consequence of the result?
Table 11.7 above indicates how to document a risk analysis. Column P is the probability of the risk occurrence, on a scale of 1 (not very probable) to 5 (almost certain to occur). Alternatively, you can also express the probability by the rating of low, medium or high. Column C describes the consequence if the risk becomes a problem, expressed on a scale of 1 (of little consequence) to 5 (catastrophic consequence), or with a rating of low, medium or high. Column E indicates the exposure to the risk. If numerical values were used to estimate the probability of the risk and its consequence, then the exposure is equal to P × C. If relative interval rating values have been used (e.g., low, medium, high), we can estimate the risk exposure using Table 11.8.
Table 11.8 Risk Classification Grid
| Consequence | |||
| Probability | Low | Medium | High |
| Low | Low | Low | Medium |
| Medium | Low | Medium | High |
| High | Medium | High | High |
Source: Adapted from Wiegers (1998) [WIE 98].
Figure 11.8 presents a risk description template originally presented by Wiegers in [WIE 98]. It contains more information than the risk classification grid above. A field entitled “first indicator” documents the trigger that would cause this risk to become a problem. Another field identifies the individual responsible for acting on that risk and another indicates the time when risk mitigation should be completed.
Figure 11.8 A risk documentation template [WIE 98].
11.3.1.3 Risk Prioritization Activity
This activity produces a prioritized list of risks, such as the top-ten risks, that have been analyzed. Priority-setting techniques include: risk analysis, cost-benefit analysis, and the Delphi technique [BOE 91]. Regarding priorities, there are two simple questions to be asked:
- – what is going to hurt the project the most?
- – what is going to hurt the project the soonest?
If numerical values are used to determine the likelihood and consequence of a risk, one can then calculate risk exposure using the simple calculation “probability × consequence.” For example, for a probability of 3 and a consequence of 4, we will get a priority of 12. You could add a column, to the right of the grid presented in Table 11.7 to document the resulting exposure. Having estimated the exposure of each risk, it is now easy to prioritize them.
Now that the risks have been assessed and prioritized, we can proceed with the risk control activities.
11.3.2 Risk Control Step
The risk control step involves three main activities: RMP, risk resolution, and risk monitoring.
11.3.2.1 RMP activity
Risk management planning is the selection of a risk management technique for each identified risk within the context of the project. The techniques and tools include risk control lists, cost–benefit analysis and the description of the contents of the RMP according to the standard used.
Each risk identified has its own mini action plan. The RMP consists in integrating each mini action plan. Some parts of the RMP may appear in other documents such as the project plan. For a large project, the table of contents of a plan may include the elements listed in the table of contents of ISO 16085 presented above. The RMP, once approved, is baselined and stored in the organizational repository. The RMP and it should follow, like any other document of a project, the organization's configuration management process.
The column “risk mitigation” in Table 11.7 (also called “risk reduction”) indicates, for each negative risk, an approach to avoid the risk, to transfer, check, accept, or monitor the risk. The risk mitigation actions must produce tangible results that will determine whether the risk of exposure changes [SEI 10a]:
- – risk avoidance refers to the elimination of the risk from the project. For example, this can be done by not developing a risk component;
- – risk transfer is to divert to a third party, such as a supplier, the risk and the responsibility of its resolution. Transferring the risk does not eliminate it;
- – risk acceptance means that no action will be taken regarding the risk;
- – risk control means that certain actions are taken, between now and the time when the risk can occur by reducing the probability and/or impact or its consequence;
- – risk monitoring means observing and periodically re-evaluating the risk to detect changes in parameters.
Contingency planning means that preparations are made before the time when the risk can materialize, which define actions to be taken should the risk occur.
It is possible to add additional columns to the grid. For example, one might add the name of the person responsible for a risk to the right of the grid shown in Table 11.9. We could add a column to the right to indicate when the risk mitigation actions should have been established. Finally, we could add a column to show the status of the actions to reduce each risk as follows: P for in progress and C for completed.
Table 11.9 Expanded Risk Document Grid [WIE 98]
| Risk | Person | Risk mitigation | ||||||
| identification | Risk | Risk | responsible | completion | Status | |||
| number | description | P | C | E | mitigation | for risk | date | (P/C) |
| 1 | ||||||||
| 2 | ||||||||
| 3 |
For small projects, we could add, as an annex, the form illustrated in Figure 11.9 or the individual risk forms illustrated in Figure 11.8.
Figure 11.9 Grid illustrating risk exposure using three categories.
11.3.2.2 Risk Resolution Activity
The resolution of risks produces a situation in which identified risks are eliminated or otherwise resolved (e.g., by easing requirements) using the risk mitigation techniques documented in the project plan developed in the previous activity.
11.3.2.3 Risk Monitoring Activity
Monitoring risk involves monitoring project progress to address risk factors and take corrective action if necessary. The techniques and tools include: risk audits, gap analysis and trends, project reviews, monitoring milestones, and the list of the most significant risks [BOE 91]. We can use the form illustrated in Figure 11.9 or the set of individual forms of Figure 11.8 during the project progress meetings to track each risk and, if necessary, make changes to documents (e.g., probability, consequence, status).
11.3.3 Lessons Learned Activity
The elements of the risk management process, as shown in Figure 11.7, can be improved by conducting lessons learned sessions, as discussed in Chapter 5, at the end of a development project to identify weaknesses and propose possible improvements.
When holding a lessons learned session, the project manager and his team could discuss the project risks, the risks identified and described in the project plan and unidentified risks that came up during the project.
Regarding the risks identified (i.e., the known risks) and documented in the project plan, the team could discuss the probabilities, consequences, and mitigation measures that were satisfactory. Otherwise, the team could suggest improvements to the process.
For risks that were not identified (i.e., the unknown risks), were these risks on the list of potential risks of the project management process but were not identified during the development of the project plan? In this case, the project manager should analyze the assumptions he used for the preparation of the project plan and decide whether to amend the risk management tasks of the planning process. If the risks that were not identified were not on the list of potential risks, the project manager should perhaps add them to the list of risks. The new list of risks will be used in a future project.
11.4 Risk Management Roles
A risk management process requires the participation of several project stakeholders such as the project manager, the development team, marketing and customers. In a small entity, one person may play many of these roles:
- – the project manager is responsible for managing the risks associated with the development and maintenance of the system and ensures that risk management is conducted in accordance with the organizational process;
- – the risk manager (a role of large organizations or projects): the project manager of a large project can choose to play this role. This role must perform the risk management process and serve as a “facilitator” for the risk analysis activity with other stakeholders;
- – developers participate in the risk identification, analysis, documentation, and monitoring;
- – SQA periodically reviews risk management activities to ensure that they are carried out as they were planned by the project. The SQA specialist can also participate in the risk identification and lessons learned by playing the role of a facilitator;
- – configuration management can also play a role in risk monitoring and reporting. For example, the CM manager could be responsible for determining the risk status;
- – risk monitor or risk owner is the person responsible for monitoring the evolution of a specific risk.
11.5 Measurement and Risk Management
Assessing risk requires the following measures at the least:
- – probability: a measure of the likelihood of a threat occurring. For example, this measure can be a value from 0 to 5 or 0 to 10 or a qualitative value such as low, medium, high. Table 11.10 shows an example;
- – a measure of the extent of the potential impact or consequence of the risk: a measure of the loss than could occur if a threat materializes. For example, this measure can be a value from 0 to 5 or 0 to 10 or also a qualitative value, such as low, medium, or high. Table 11.11 shows an example;
- – risk exposure: a measure of the magnitude of a risk based on the probability and potential impact. It is easier to provide this if the probability and impact measures were calculated numerically. Otherwise, a grid, as shown in Figure 11.9, may be used to illustrate the risk exposure using a scale of low, medium or high. The portion of the grid which is located at the bottom left shows the low exposure area, the upper right region indicates high exposure, and the intermediate zone indicates an average exposure to risk.
Table 11.10 Example of Risk Probability Categories
| Value | |
| 1 | Not likely |
| 2 | Low likelihood |
| 3 | Likely |
| 4 | Highly likely |
| 5 | Near certainty |
Source: Adapted from Shepehrd (1997) [SHE 97].
Table 11.11 Example of Risk Consequence Categories
| Level | Technical | Schedule | Cost |
| 1 | Minimal or no impact | Minimal or no impact | Minimal or no impact |
| 2 | Minor performance shortfall, same approach retained | Additional activities required; able to meet key dates | Budget increase or unit production cost increase <1% |
| 3 | Moderate performance shortfall, but workarounds available | Minor schedule slippage; will miss need date | Budget increase or unit production cost increase <5% |
| 4 | Unacceptable, but workarounds available | Critical path affected | Budget increase or unit production cost increase <10% |
| 5 | Unacceptable, no alternatives exist | Cannot achieve key milestone | Budget increase or production cost increase >10% |
Source: Adapted from Shepehrd (1997) [SHE 97].
Figure 11.10 shows three examples of risks. Risk 1 is a medium risk since it is likely that the schedule is acceptable, risk 2 is low and risk 3 is a high risk.
Figure 11.10 Example of three project risks.
Source: Adapted from Shepehrd (1997) [SHE 97].
For low risks, usually we will not take any specific action. For medium risks, close monitoring will be sufficient, while for high risk, action must be taken as soon as possible.
We can also measure different risk management elements during a project, for example:
- – the number of risks identified;
- – the number of active risks;
- – the number of risks by exposure category (e.g., low, average, high);
- – the effort for risk management (e.g., in person-hours);
- – the number of risks identified, managed, and monitored;
- – the number of risks that were not identified;
- – the number of risk management process audits;
- – the number of closed risks since the project was started compared to the number initially identified;
- – the percentage of the budget dedicated to risk management activities.
Note that you should be prudent with these measurements since risk management is already a delicate subject to manage.
The following text box describes an industry application of a risk management approach.
11.6 Human Factors and Risk Management
Risk management is not a purely rational process. It includes a strong human and cultural component regarding motivations, perceptions, interactions between roles, communication, decision making, and risk tolerance.
For example, a corporate culture that values and rewards heroes and fire fighters and that does not value people who can solve issues proactively before they become problems, will have a hard time implementing an effective risk management process. To change this culture, the organization will need to reward people who know how to identify, address, and avoid risks before they become problems. The organization will also have to accept that occasionally there will be fires to put out and that firefighters will still be required.
The next text box presents some guidelines that allow a person to anonymously report a risk in an organization that is wary of risks.
Table 11.12 lists some attitudes that, if present in an organization, will make it difficult to implement an effective risk management process.
Table 11.12 Attitudes that Make it Difficult to Implement Risk Management
| We blame someone who has committed an error. |
| Information is not shared because information is power. |
| Lone rangers and fire fighters are promoted. |
| Failure is not an option and should not be possible. |
| We never talk about risks. |
| We look for scapegoats. |
| We never reflect on past projects. |
| We shoot the messenger when he comes with bad news. |
| We never address real problems in meetings. |
| We hide risks, because no contingency budget was approved. |
| We (i.e. management) do not want to hear about problems if the solution does not come with it. “Don't bring me problems, bring me solutions!” |
| We make decisions only when the problem has erupted into a crisis. |
| A quiet meeting is a sign that everything is under control. |
| We think that success comes with hard work. |
| We believe in miracles. |
The next text box presents a list of excuses used to avoid implementing risk management.
11.7 Success Factors
The following text box lists some of the factors that help or prevent effective risk management.
When developing a project plan, any assumption that we make, often unconsciously, is a risk that we accept. For example, an assumption used by most organizations when they underestimate effort is that the productivity of their programmers is above average and therefore will require less effort. If you consider your developers to be above average, while they are realistically average or even below average, your risk increases right at the beginning of a project.
11.8 Conclusion
We can summarize risk management as taking into account these basic rules:
- – always have an alternative;
- – be prepared to manage crises;
- – increase the probability of acceptable results;
- – reduce the impact of less desirable results;
- – reduce the probability of the risk event itself;
- – identify parallel tracks, delivery deadlines, decision points, and action plans;
- – have a clear mechanism to solve problems and communicate results.
Do not forget that despite it all, risks can also be opportunities.
11.9 Further Reading
- CHARETTE R. N. Software Engineering Risk Analysis and Management. McGraw-Hill, New York, 1989.
- CHARETTE R. N. Applications Strategies for Risk Management. McGraw-Hill, New York, 1990.
- DEMARCO T. and LISTER T. Waltzing with Bears: Managing Risk on Software Projects, Dorset House Publishing, New York, 2003.
- MCCONNELL S. Rapid Development: Taming Wild Software Schedules. Microsoft Press, Redmond, WA, 1996.
- HALL E. M. Managing Risk – Methods for Software Systems Development. Addison-Wesley, London, UK, 1998.
- OULD M. Strategies for Software Engineering: The Management of Risk and Quality. John Wiley & Sons, Ltd, Chichester, UK, 1990.
- POULIN L. Reducing Risk in Software Process Improvement. Auerbach Publications, Boca Raton, FL, 2005.
11.10 Exercises
-
Develop and draw, using the ETVX notation and the ISO 16085 standard, a risk management process for an organization with fewer than 10 employees.
-
List five risks for each phase of a typical development project.
-
List five potential risks when reusing components already developed in your organization.
-
List five potential risks when a project intends to acquire Commercial Off-The-Shelf (COTS) software components.
-
An emergency plan is a plan that is implemented when a risk becomes a problem. Give examples of emergency plans.
-
A supplier cannot deliver the software at the required level of reliability and consequently the reliability of the system may not meet the customer performance specifications. Describe the possible risk management measures to be taken in such a case.
-
The interface with a new control device is not yet defined. The software driver may take longer to develop than initially estimated. Describe the possible risk management measures to be taken in such a case.