After reading this chapter you will be able to:
This chapter is devoted to the use of concepts and practices presented in this book to implement a software quality assurance plan (SQAP). Figure 13.1, adapted from Daniel Galin's house of software quality, connects all of the book's concepts as components that must come together to achieve software quality.
Before diving into the development of a SQAP, it is worth reviewing the definition of SQA as described in the following text box.
IEEE 730 states that the term “SQA function” is not to be interpreted as a particular person, tool, document, job title, or a specific group dedicated to SQA, regardless of how that function is staffed, organized, or executed. The SQA function's responsibility is to produce and collect evidence that forms the basis for giving a justified statement of confidence that the software product conforms to its established requirements [IEE 14].
Two clauses of IEEE 730 are dedicated to the planning and execution of the SQAP: clause 5.3.3 entitled “Document SQA planning,” and clause 5.3.4 entitled “Execute the SQA Plan.” In the first part of this chapter, we present the SQA planning, and in the second part, we will present the execution of the SQAP.
Thirteen mandatory tasks that are part of clause 5.3.3 describe what has to be done by the SQA function during the software planning stage of a project. IEEE 730 states that SQA activities are planned and executed in a manner that is commensurate with product risk—the higher the product risk, the greater the breadth and depth of SQA activities.
As can be seen in the following text box, the IEEE 730 standard [IEE 14] describes the normative outline of a SQAP.
One task included in clause 5.3.3, task 12, is quite clear about the content and the development of the SQAP [IEE 14]: address all of the topics listed in the normative SQA plan outline. Every section in the plan is to be included. The topics of the SQAP are normative but the section names and order are informative. If a section in the outline is not applicable to a given project, a placeholder for that section may be included along with a justification for why the topic is not applicable. This clause must be respected if an organization wants to claim conformance to IEEE 730. If an organization does not have to conform to IEEE 730, it may use this chapter as a set of guidelines in the development of a SQAP commensurate with the risks of the product to be developed.
Preparing and obtaining sign off on a SQAP is often the project manager's or the quality assurance manager's responsibility. SQA can help by offering a standardized template, examples, and explanations that can assist with this task. Organizations with an existing quality assurance function should have developed a SQAP template adapted to their internal methodology. A quality plan, which describes the quality activities and tasks to be executed during a software project, should be created from this template, approved by stakeholders, and kept up to date throughout the project. Exemplary SQAPs (i.e., from previous projects) can be made available in the life cycle process library to facilitate the development of a project-specific SQAP and to show newcomers examples of past projects in order to help them understand the typical content required.
The following sections present, in more detail, the contents of each section of the SQAP according to Annex C of the IEEE 730 standard, and refer to specific chapters and sections of this book where detailed information has already been provided on the topic. Annex C also provides “suggested inputs” to help with the preparation and execution of a SQAP.
This first section of a SQAP should describe the purpose, objectives, and scope of quality assurance activities that will be undertaken, including waivers obtained if present. This section should also explicitly identify processes and software life cycle products covered by quality assurance activities. It is a good idea to summarize which business model is associated with the specific project. You can find information about the different IT business models and their influence on quality assurance practices in section 1.6 of this book.
Here are questions that the SQA function can ask during the project planning phase to help refine the purpose and scope of SQA for a new software project (IEEE 730, Table C.3):
This section of the SQAP ensures that abbreviations and acronyms are explicitly stated. Refer to Chapter 1 to review information about the terminology that should be used consistently by software engineers when discussing quality. This is important because using recognized and standardized terminology that is fully aligned with approved software engineering standards will ensure that you can use your body of knowledge when a problem occurs. If a debate occurs, during final testing and delivery, about the meaning of a deliverable or a responsibility, you will always be able to fall back on your standards and body of knowledge to explain the intended meanings and solve these issues rapidly. It also ensures definitions, acronyms, and abbreviations are clarified for the project participants.
This section of the SQAP identifies all applicable standards, industry-specific regulations and compliance clauses of contracts, other documents referenced by the SQAP, and any relevant supporting documents. Supporting documents may include applicable professional, industry, government, corporate, organizational, and project-specific references. Here are the questions that should be answered during the project-planning phase (IEEE 730, Table C.4):
The configuration management chapter of this book has presented how to refer, safeguard and manage successive versions of key project documents and life cycle artifacts. Documents referred to may include industry, legislative, or contractual documents pertaining to the project. The SQAP should also identify the origin of the project's requirements such as contracts, specifications documents, or deliverables list. Key project documentation and mandatory deliverables should be described with more detail, such as:
The benefit of this section of the SQAP is that by trying to produce Table 13.1 at the beginning of a project, the project manager will be forced to: (1) identify mandatory deliverables and (2) see if all the resources are available for the task ahead.
Table 13.1 Example of Key Documents and Mandatory Deliverables Checklist for the Project
Document name | File name | Location | Reference template used | Authors | Approbation |
Project plan | Project_143_ plan.doc | C :Project_ 143-Plan | Yes | M. Smith | A.Anderson |
Functional specifications | Project_143_ specs.doc | C :Project_ 143-Specs | Yes | D. Connor | B.Thomas, P.Rodriguez |
… | … | … | … | … | … |
It becomes a useful checklist during project planning. Table 13.1 shows an example of the description of two mandatory software project deliverables. Organizations must decide on their minimum list of mandatory deliverables to be produced during a software project to ensure that the software meets the internal requirements of their internal software development methodology.
When reviewing the SQAP and during project reviews, the following questions should be asked:
Software project organization and project management, as a whole, are topics that have not been covered by this book. They are important topics that impact software quality.
This section of the SQAP aims at clarifying the parties responsible for performing SQA, for a project, and seeks to present the interrelationship with the project management team. A functional organization chart often represents these relationships. If subcontractors are involved, the relationships and information flow between SQA and each subcontractor should also be shown. By clarifying relationships in this section of the SQAP it will be possible to see if the required roles and responsibilities are present. Here are the questions that SQA can ask during the project-planning phase concerning this topic (IEEE 730, Table C.5):
It is therefore suggested that the organizational structure of the software project be described and that the role of each participant is identified in this section of the plan. Figure 13.2 shows individual part of the management committee, project committee, and those supporting the project that will be asked to play a role.
Creating a visual representation of a project organization will quickly highlight if the numbers of individuals that have been assigned to the project are sufficient, if they are located at the right place and if they are qualified to do the work.
A RACI chart can also be used to describe each individual role. Rarely does everyone on a team have the same understanding of who is responsible for what. A RACI chart, also known as a RACI matrix, clarifies roles and responsibilities, making sure that nothing is forgotten. RACI charts also eliminate redundant activities and confusion by assigning clear ownership for each task or decision. It is important to identify early if a person is playing too many roles or, conversely, if a role is currently vacant/unassigned. Table 13.2 is another simple way of documenting roles and responsibilities of individuals in a software project.
Table 13.2 Specifying the Names of Individuals Who are Involved in a Project
Role | Assigned to | Responsibilities |
Project committee | A. Lopez (Pilot), G. Wright (IT Responsible), M. Thomas (SQA), P. Smith (IT expert) | Follow the progress, address issues, set project directions and priorities, authorize budgets and changes. |
Key end-user | P. Clark, H. Johnson | Validate requirements, functional experts, conduct functional acceptance of system. |
This section of the SQAP should also specify the degree of independence between the organization performing the SQA function and the project team members. There are three parameters that can be used to define independence: (1) technical independence, (2) managerial independence, and (3) financial independence. Technical independence requires that SQA utilize personnel who are not involved in the development of the system or its elements. SQA forms its own assessments of all project activities. Technical independence is an important method to detect subtle errors overlooked by those too close to the solution. Managerial independence requires that responsibility for SQA be vested in an organization separate from the software development and program management organizations. Managerial independence also means that SQA independently selects segments of software to analyze and test, chooses techniques, defines the schedule of SQA activities, and selects the specific technical issues and problems to act upon. The SQA effort provides its findings in a timely fashion simultaneously to both the software development and program management organizations. Financial independence requires that control of the SQAP budget be vested in an organization independent of the software development organization. This independence prevents situations where SQA cannot complete its activities because funds have been diverted or adverse financial pressures or influences have been exerted.
The chapter on risk management has presented how risk relates to SQA. Software product risk refers to the inherent risks associated with the use of the software product (e.g., safety risk, financial risk, security risk). Software product risk is different from project management risk, which is addressed later in this plan. Specific V&V techniques may be required to address software product risk. Table 13.3 lists the questions and suggested inputs, such as a RMP, that SQA can discuss during the project-planning phase (IEEE 730, Table C.7).
Table 13.3 Questions and Suggested Inputs Related to Software Product Risk to Consider Asking During Project Planning Phase [IEE 14]
Questions | Suggested inputs |
|
|
To explain the extent of risk management activities, this section of the SQAP should clearly state what level of integrity the software being worked on is at. Criticality analysis of software components, which includes four levels, was presented in the chapter on V&V. The SQAP must ensure that the tasks and activities identified are carried out in a way that is commensurate with the level of criticality of the assigned software. Depending on the software criticality, projects apply more or less strict engineering and SQA requirements. This important information will allow the reader to assess the number of compensating provisions required to mitigate the consequences of software failure.
This section of the SQAP describes tools to be used by SQA to perform specific tasks. These may include a variety of software tools to be used as part of the SQA process. Appropriate acquisition, documentation, training, support, validation, and qualification information for each tool is included in the SQAP. Here are the questions that SQA can ask during the project-planning phase:
Regarding the tools, do not forget to specify the version of the tools used for a given project. It is also in this section of the plan that you should describe the tools and techniques that will be identified by the team for this project.
Chapter 4 of the book described software engineering standards and how they relate to SQA. This section of the SQAP identifies standards, practices, and conventions to be used in performing activities and tasks and for creating outcomes:
Describe the techniques and methods to be used for the project. Chapter 9 discussed how policies, processes, and procedures are used by a project. For software projects involving external suppliers, it is important to clarify which methodology (e.g., Oracle Unified Method, IBM Rational Unified Process, Scrum) will be used and also to clarify the obligations to produce certain mandatory deliverables (see Table 13.2).
This section also lists which standards, practices, and conventions apply to this project. It will specify the stage of the life cycle, the list of intermediate deliverables as well the standards, guides, or conventions used to ensure that quality will be met during reviews, inspections, and final acceptance steps of the project. We have discussed many standards and models in Chapter 4.
Table 13.4 provides an example of a simple presentation of these concepts.
Table 13.4 Reference Table for Standards, Practices, and Conventions
Step of the life cycle | Intermediary deliverable | Standard, practice, orconvention |
Planning | Project plan | PMI - PMBOK® Guide |
Planning | SQAP | IEEE 730 |
Programming | Source code | Java programming rules revised April, 20 1999 |
Transition to production | Technical documentation | A local production criteria and checklist |
By specifying these items in advance, we facilitate the task of reviewing the quality plan and project reviews for the project manager and SQA specialist.
This section includes estimates of the effort required to complete the activities, tasks, and outcomes as defined in the SQAP. This section identifies appropriately qualified SQA personnel and defines their specific responsibilities and authority within the context of the project.
This section also identifies additional SQA resources, including facilities, lab space, and special procedural requirements (e.g., security access rights and documentation control) that are required to perform SQA activities. This section should also include a list of critical SQA project milestones and a schedule of planned SQA activities, tasks, and outcomes. Here are the questions that SQA can ask during the project planning phase:
Typically, the estimates describe the software activities pretty well (i.e., feasibility, requirements, design, construction, testing, and delivery). But, have all the efforts associated with the cost of quality been considered in these estimates and the schedule (prevention costs, appraisal costs, failure costs)? Refer to section 2.2 on cost of quality for details.
For the assessment of the project schedule, this section of the SQAP should describe the project milestones and the planned SQA in the project schedule. Specifically, it indicates the deliverables that will be produced by the SQA activities required by the project manager and the project committee. A good schedule reflects all project activities and shows the individual assignment of personnel. To verify whether a scheduled assignment is realistic and reviewed, an assignment email could be sent to individuals during planning. A letter of assignment (also called a work assignment email) is to formally contact a project resource asking:
This simple approach validates the information described by the schedule. It works especially well with team members that do not report directly to the project manager. Finally, this section of the SQAP is also used to describe the qualifications of staff who will perform SQA activities during the project and to clarify their responsibilities, their degree of independence, and their authority to perform reviews, inspections, and ask for corrections. The plan should also identify additional resources, including facilities, tools, test labs and other requirements (e.g., security permissions and code/document repository access rights).
Every project and its plan is conceived and developed based on a set of hypotheses, scenarios, or assumptions. Assumptions analysis explores the validity of assumptions as they apply to the project. It identifies risks to the project from inaccuracy, instability, inconsistency, or incompleteness of assumptions [PMI 13]. The PMBOK® Guide recommends that assumptions made in developing the activity duration estimate, such as skill levels and availability, as well as a basis of estimates for durations, be documented in project documents. When developing a SQAP, assumptions should be documented and analyzed when estimating effort, resources, and schedules.
Product assurance activities provide confidence that software products are developed in conformance to established product requirements, project plans, and contractual requirements. An important aspect of SQA is the establishment of confidence in the quality of the software products produced by the project. These products include not only the software and related documentation but also the plans associated with the development, operation, support, maintenance, and retirement of the software. A product may also be a software service provided to the acquirer. Product assurance activities may include SQA personnel participating in project technical reviews, software development document reviews, and software validation testing.
The outcome of the product assurance activities provides evidence that the software services, products, and any related documentation are identified in and comply with the contract and any non-conformances are identified and addressed. Product assurance is comprised of three activities:
Product assurance requires that some measurement be performed on software products. This section of the SQAP should also identify activities and tasks for evaluating whether the measurements objectively demonstrate the quality of the products in accordance with established standards and processes. Section 3.3 of the book describes a process to formally define software quality requirements in a project so that the measurement can be used during acceptance. Chapter 10 also presents measurement topics.
Similar to the product assurance, process assurance aims to ensure that the processes used by the project are appropriate, depending on the level of integrity, and are followed:
This section also requires the use of reviews and audits in the project. The reviews were presented in Chapter 5 and audits in Chapter 6. Table 13.4 provides an example of a simple presentation of reviews and audits for the project. Configuration management concepts were also discussed in Chapter 8.
Process assurance requires that some measurement be done on software processes. This section of the SQAP should identify activities and tasks for evaluating whether the measurements support effective management of the processes in accordance with established standards and processes (e.g., appropriate set of measures identified and planned, a data collection and analysis process is selected, measurement is consistent with product risk and overall quality goals). Measurement was discussed in Chapter 10.
Finally, this section of the plan should assess staff knowledge and skill. It is necessary that the plan outlines whether the staff assigned to the project has the qualifications and training necessary. If a training plan is to be prepared and executed it typically has two specific audiences: (1) technical specialists who will need specific training and (2) user representatives who will perform the final acceptance as well as future end-users.
In this part of the SQA plan, describe the scope of training needs and the scope covered by the project. Here are the questions that SQA can ask during the project planning phase:
This section of the SQAP identifies any additional considerations, such as SQA processes that support both Project Management and Organizational Quality Management, that are not described elsewhere. The following topics are included in this section of the SQAP:
- describe the software acquisition process for the project;
- identify the reviews that will be conducted by the supplier during the project;
- identify the appropriate contract type;
- specify which contract template will be used and the planned contract reviews.
- select and describe the quality characteristics and sub-characteristics to be evaluated;
- specify how this will be measured and clarify the software attributes to be measured;
- set a quality objective target for the project;
- describe, with an example, how the calculation is done;
- explain how and what tools will be used to evaluate this measure at the acceptance stage.
- project progress for each of the five project dimensions of a software project (e.g., schedule, staff, features, cost and quality) as presented in section 2.4 of Chapter 2 and referred to by the PSM in Chapter 11;
- defects density and reviews/audits of non-conformances correction status.
Table 13.5 Overview of Planned Reviews of a Project
Review name | Objective | Life cycle phase | Intermediary product | Type ofreview |
Requirements | Ensure the completeness/testability of requirements | Requirements stage | Functional specifications | Document reviews |
Architecture and design | Ensure the maintainability and traceability of design to requirements | Architecture and design stage | Design documents | Document reviews |
Source code | Ensure the conformity to local programming | Programming stage | Source code & unit test plans and results | Code and documents inspection |
Quality audit just before system tests | Progress and readiness of products | Integration tests stage | Overall project | Quality audit |
- accept the document as is (acceptance);
- accept the document with minor changes (conditional acceptance);
- repeat the review when the significant changes agreed to in the review meeting have been incorporated.
- 100% of the functionality approved in the requirements document has been delivered and there is no level 1 or level 2 incident report pending a fix;
- The infrastructure and maintenance/support groups have no pending level 1 or 2 incident reports. If there is a level 1 or level 2 incident report, they will have to be addressed before final acceptance of the software.
Table 13.6 Defect Severity Classification Example
Category | Description |
1 | A defect that prevents the project from testing further as this defect needs to be corrected before going ahead. A solution must be found immediately and corrected before continuing to test this functionality. |
2 | An important defect that prevents the project from testing parts of a functionality but other parts can continue to be tested. A solution needs to be found in a reasonable delay. |
3 | A defect that does not cause a major problem for testing to continue. A solution is required but can be placed in a priority list to be treated after category 1 and 2 are done. |
4 | A minor defect to be placed in the list of things to do. |
This section of the SQAP includes activities and tasks for analysis, identification, collection, filing, maintenance, and disposition of quality records. Quality records document that activities were performed in accordance with project plans and the contract. These records enable information sharing and support analysis to identify problems, causes, and eventually result in product and process improvements. The project manager and/or the SQA manager should ask the following questions related to the analysis, identification, collection, filing, maintaining, and disposition of SQA records during the project planning phase:
Once a SQAP has been developed and approved, the project has to execute it. The purpose of clause 5.3.4 of IEEE 730 is to “Execute the SQA Plan” in coordination with the project manager, the project team, and organizational quality management. The tasks linked to the execution of the SQAP are quite explicit. To accomplish this activity, the SQA function shall perform the following tasks [IEE 14]:
Annex C of IEEE 730 provides, for each section of the SQAP, a list of questions when executing SQA project activities described in the SQAP. Table 13.7 presents an example of such a table for the questions and suggested inputs related to software product risk to consider asking during the project executing phase.
Table 13.7 Questions and Suggested Inputs Related to Software Product Risk to Consider Asking During Project Executing Phase [IEE 14])
Questions | Suggested inputs |
|
|
Task 3 of clause 5.3.4 states that the SQA function shall revise the SQAP in response to project changes. A project is executed in an organization where processes, such as configuration management, have been documented and implemented. The SQAP must follow the organizational configuration management process when revising and updating the SQAP. Usually, a SQAP will have a table, called a revision or history table, listing the approval and revision information. The following text box briefly describes the tasks when producing an updated version of a SQAP.
The SQAP is the cornerstone of any software project aiming at producing a quality product for external or internal customers. The SQAP pulls together all quality concerns of an organization, its policies, its people, its processes, and tools in producing quality products while building a competitive software capability.
While supporting the development of quality products, a robust SQAP should minimize avoidable rework. In Chapter 2, we presented the concept of cost of quality, and in many chapters we have illustrated how SQA could help in reducing the cost of rework. Hopefully, the following statement made by Dr. Robert Charette will no longer be true for organizations that have implemented the SQA practices presented in this book.
Name five topics to be addressed in a SQAP.
Describe the review and acceptance of documents approach proposed to ensure quality.
Describe the process of acceptance of an acceptance test and how to use the categories of defects to ensure quality of delivery.
How important is the description of the criticality of the software at the beginning of a SQA plan?
Develop a checklist that will allow you to ensure that the SQA plan conforms to IEEE 730.
What is the difference between process assurance and product assurance?
What could be the consequences of not having a SQA plan for your organization?
What could be the consequences of not requiring a SQA plan from your supplier?
3.145.151.26