Process and Product Quality Assurance: A Support Process Area at Maturity Level 2
Purpose
The purpose of Process and Product Quality Assurance (PPQA) is to provide staff and management with objective insight into processes and associated work products.
Tip
PPQA is often referred to as the “eyes and ears” of the organization. It provides the insights needed so that the organization can determine whether its policies, practices, and processes are being followed; whether they are effective; and whether the work products meet the needs of the organization or project.
Introductory Notes
The Process and Product Quality Assurance process area involves the following activities:
• Objectively evaluating performed processes and work products against applicable process descriptions, standards, and procedures
• Identifying and documenting noncompliance issues
• Providing feedback to project staff and managers on the results of quality assurance activities
• Ensuring that noncompliance issues are addressed
Tip
The phrase process descriptions, standards, and procedures is used in PPQA (and in GP 2.9) to represent management’s expectations about how project work will be performed. Applicable process descriptions, standards, and procedures are typically identified during project planning.
The Process and Product Quality Assurance process area supports the delivery of high-quality products by providing project staff and managers at all levels with appropriate visibility into, and feedback on, processes and associated work products throughout the life of the project.
The practices in the Process and Product Quality Assurance process area ensure that planned processes are implemented, while the practices in the Acquisition Verification process area ensure that specified requirements are satisfied. These two process areas can on occasion address the same work product but from different perspectives. Projects should take advantage of the overlap to minimize duplication of effort while taking care to maintain separate perspectives.
Objectivity in process and product quality assurance evaluations is critical to the success of the project. (See the definition of “objectively evaluate” in the glossary.) Objectivity is achieved by both independence and the use of criteria. A combination of methods providing evaluations against criteria by those who do not produce the work product is often used. Less formal methods can be used to provide broad day-to-day coverage. More formal methods can be used periodically to assure objectivity.
Hint
During a SCAMPI appraisal, the appraisal team may not be able to judge the effectiveness of your processes or work products, but they can determine whether you have the ability to judge the effectiveness of your processes or work products and whether you are taking appropriate action based on these assessments.
Traditionally, a quality assurance group that is independent of the project provides objectivity. However, another approach may be appropriate in some organizations to implement the process and product quality assurance role without that kind of independence.
Tip
Most acquirers don’t use an independent quality assurance function to objectively evaluate process and product quality. In such circumstances, using objective criteria to evaluate quality and allowing for escalation of issues without retribution becomes critical.
Tip
For less mature acquisition organizations, it may be best to use personnel outside the project team, perhaps from another acquisition project, to help perform more formal audits of critical processes or work products.
If quality assurance is embedded in the process, several issues should be addressed to ensure objectivity. Everyone performing quality assurance activities should be trained in quality assurance. Those who perform quality assurance activities for a work product should be separate from those who are directly involved in developing or maintaining the work product. An independent reporting channel to the appropriate level of organizational management should be available so that noncompliance issues can be escalated as necessary.
Tip
Because most acquisition project teams are very small, team members tend to fill multiple roles and quality assurance activities are embedded within the process. Given this arrangement, it is important to ensure that objectivity is achieved.
Tip
Acquisition project leadership must create an environment in which team members are encouraged to identify and fix process and product defects.
Quality assurance should begin in the early phases of a project to establish plans, processes, standards, and procedures that will add value to the project and satisfy the requirements of the project and organizational policies. Those who perform quality assurance activities participate in establishing plans, processes, standards, and procedures to ensure that they fit project needs and that they will be usable for performing quality assurance evaluations. In addition, processes and associated work products to be evaluated during the project are designated. This designation can be based on sampling or on objective criteria that are consistent with organizational policies, project requirements, and project needs.
Hint
Establish your quality assurance approach early in the project and continually focus on identifying constraints that may keep you from achieving project objectives.
Hint
Use a risk-based approach when designating which processes and work products to evaluate.
When noncompliance issues are identified, they are first addressed in the project and resolved there if possible. Noncompliance issues that cannot be resolved in the project are escalated to an appropriate level of management for resolution.
This process area applies to evaluations of project activities and work products, and to organizational (e.g., process group, organizational training) activities and work products. For organizational activities and work products, the term “project” should be appropriately interpreted.
Tip
Limit the appraisal scope to those supplier processes that are most critical to the current phase of the project. Use Class B or Class C (nonbenchmarking) methods to help identify and mitigate project risk.
X-Ref
For more information about interpreting your suppliers’ CMMI achievements, see “Understanding and Leveraging a Supplier’s CMMI Efforts: A Guidebook for Acquirers” at www.sei.cmu.edu/library/abstracts/reports/07tr004.cfm.
Related Process Areas
Refer to the Acquisition Verification process area for more information about ensuring that selected work products meet their specified requirements.
Hint
Determine the effectiveness of your processes and work products by evaluating adherence to objective criteria.
Specific Practices by Goal
SG 1 Objectively Evaluate Processes and Work Products
Adherence of the performed process and associated work products to applicable process descriptions, standards, and procedures is objectively evaluated.
Hint
Regularly ask your customers and suppliers how they perceive the effectiveness and efficiency of your processes and the quality of your work products as contributing to their own success.
SP 1.1 Objectively Evaluate Processes
Objectively evaluate selected performed processes against applicable process descriptions, standards, and procedures.
Objectivity in quality assurance evaluations is critical to the success of the project. A description of the quality assurance reporting chain and how it ensures objectivity should be defined.
Tip
Some of the most critical processes to evaluate are those that cross organizational boundaries. Acquirer-to-customer, acquirer-to-supplier, and acquirer-to-acquirer processes in a system-of-systems acquisition are good candidates to evaluate regularly.
Example Work Products
1. Evaluation reports
2. Noncompliance reports
3. Corrective actions
Subpractices
1. Promote an environment (created as part of project management) that encourages staff participation in identifying and reporting quality issues.
Tip
Quality is everyone’s job. It is important that everyone in the organization be comfortable identifying and openly discussing quality concerns.
2. Establish and maintain clearly stated criteria for evaluations.
The intent of this subpractice is to provide criteria, based on business needs, such as the following:
• What will be evaluated
• When or how often a process will be evaluated
• How the evaluation will be conducted
• Who must be involved in the evaluation
Hint
The results of your quality assurance activities can identify best practices and processes that you should share with others in your organization. When working with multiple suppliers on the same project, best practices that are discovered through audit activities at one supplier location can become candidates for project-wide adoption to reduce the overall project risk.
3. Use the stated criteria to evaluate selected performed processes for adherence to process descriptions, standards, and procedures.
4. Identify each noncompliance found during the evaluation.
5. Identify lessons learned that could improve processes.
SP 1.2 Objectively Evaluate Work Products
Objectively evaluate selected work products against applicable process descriptions, standards, and procedures.
1. Evaluation reports
2. Noncompliance reports
3. Corrective actions
Hint
Resist the temptation to put total attention on the suppliers’ work products and not give “due diligence” to evaluating your own (acquirer) work products. Both must be of high quality to ensure program success.
Subpractices
1. Select work products to be evaluated based on documented sampling criteria if sampling is used.
Work products can include services produced by a process whether the recipient of the service is internal or external to the project or organization.
Hint
You can embed objective evaluations of work products within some of the verification activities—particularly peer reviews—although doing so requires care. (See the introductory notes for more information.)
2. Establish and maintain clearly stated criteria for the evaluation of selected work products.
The intent of this subpractice is to provide criteria, based on business needs, such as the following:
• What will be evaluated during the evaluation of a work product
• When or how often a work product will be evaluated
• How the evaluation will be conducted
• Who must be involved in the evaluation
Tip
Subpractices 4 through 6 recommend evaluation of work products at different times and from different perspectives. The important point is that you think broadly about which steps will give you the best objective insight during your project.
3. Use the stated criteria during evaluations of selected work products.
4. Evaluate selected work products at selected times.
Hint
Your quality assurance activities can benefit other projects in your organization. For this reason, you should share your results more broadly, outside your project team.
5. Identify each case of noncompliance found during evaluations.
6. Identify lessons learned that could improve processes.
SG 2 Provide Objective Insight
Noncompliance issues are objectively tracked and communicated, and resolution is ensured.
SP 2.1 Communicate and Resolve Noncompliance Issues
Communicate quality issues and ensure the resolution of noncompliance issues with the staff and managers.
Noncompliance issues are problems identified in evaluations that reflect a lack of adherence to applicable standards, process descriptions, or procedures. The status of noncompliance issues provides an indication of quality trends. Quality issues include noncompliance issues and trend analysis results.
Hint
Noncompliance issues may be commonplace in low maturity organizations and should be addressed at the lowest possible level. Don’t “criminalize” noncompliance by pointing fingers at the “guilty parties.”
When noncompliance issues cannot be resolved in the project, use established escalation mechanisms to ensure that the appropriate level of management can resolve the issue. Track noncompliance issues to resolution.
Example Work Products
1. Corrective action reports
2. Evaluation reports
3. Quality trends
Hint
Focus on fixing the process, not the people. Create an environment where everyone feels empowered to identify and correct errors without fear of retaliation.
Subpractices
1. Resolve each noncompliance with the appropriate members of the staff if possible.
Tip
In some cases, organizational requirements and project requirements may contradict each other. These contradictions may be valid variation points that the organization could incorporate into requirements for future projects.
2. Document noncompliance issues when they cannot be resolved in the project.
Hint
Quality assurance activities are often seen as not adding value to the work products. In reality, the status and results of quality assurance activities should be reported regularly. Also, consider reporting a “compliance percentage” rather than the number of noncompliance issues. Use of this metric may encourage friendly competition across projects and increase appreciation for the quality assurance role.
3. Escalate noncompliance issues that cannot be resolved in the project to the appropriate level of management designated to receive and act on noncompliance issues.
4. Analyze noncompliance issues to see if there are quality trends that can be identified and addressed.
5. Ensure that relevant stakeholders are aware of results of evaluations and quality trends in a timely manner.
6. Periodically review open noncompliance issues and trends with the manager designated to receive and act on noncompliance issues.
7. Track noncompliance issues to resolution.
SP 2.2 Establish Records
Establish and maintain records of quality assurance activities.
1. Evaluation logs
2. Quality assurance reports
3. Status reports of corrective actions
4. Reports of quality trends
Tip
Records provide a way to identify trends in quality assurance activities (including noncompliance issues) that allow the organization to identify where additional guidance or process changes are needed.
Subpractices
1. Record process and product quality assurance activities in sufficient detail so that status and results are known.
2. Revise the status and history of quality assurance activities as necessary.