The purpose of Process and Product Quality Assurance (PPQA) is to provide staff and management with objective insight into processes and associated work products.
Tip
PPQA is often referred to as the “eyes and ears” of the organization. It provides the insights needed so that the organization can determine whether its policies, practices, and processes are followed, are effective, and that the work products produced are meeting the needs of the organization or project.
The Process and Product Quality Assurance process area involves the following activities:
Tip
The phrase process descriptions, standards, and procedures is used in PPQA (and in GP 2.9) to represent management’s expectations as to how project work will be performed. Applicable process descriptions, standards, and procedures are typically identified during project planning.
• Objectively evaluating performed processes, work products, and services against applicable process descriptions, standards, and procedures
• Identifying and documenting noncompliance issues
• Providing feedback to project staff and managers on the results of quality assurance activities
• Ensuring that noncompliance issues are addressed
The Process and Product Quality Assurance process area supports the delivery of high-quality products and services by providing project staff and managers at all levels with appropriate visibility into, and feedback on, processes and associated work products throughout the life of the project.
Hint
During a SCAMPI, the appraisal team may not be able to judge the effectiveness of your processes or work products, but they do determine whether you have the ability to judge the effectiveness of your processes or work products and that you are taking appropriate action based on your determination.
The acquirer evaluates critical acquirer work products, acquirer processes, results of supplier process quality assurance, and supplier deliverables. For example, process and product quality assurance ensures that the solicitation package was developed using standard processes agreed to by the organization and that it conforms to all applicable policies. The acquirer may review results of supplier quality assurance activities for selected supplier processes to ensure that the supplier is following its own processes.
Typically, selected supplier processes are critical processes, such as engineering or verification processes, where the supplier is required through the supplier agreement to follow project-specified standards. In exceptional cases, the acquirer may directly perform process and product quality assurance for selected supplier processes. The acquirer and supplier periodically share quality assurance issues and findings that are of mutual interest.
The practices in the Process and Product Quality Assurance process area ensure that planned processes are implemented, while the practices in the Acquisition Verification process area ensure that specified requirements are satisfied. These two process areas may on occasion address the same work product but from different perspectives. Projects should take advantage of the overlap in order to minimize duplication of effort while taking care to maintain separate perspectives.
Objectivity in process and product quality assurance evaluations is critical to the success of the project. (See the definition of “objectively evaluate” in the glossary.) Objectivity is achieved by both independence and the use of criteria. A combination of methods providing evaluations against criteria by those not producing the work product is often used. Less formal methods can be used to provide broad day-to-day coverage. More formal methods can be used periodically to assure objectivity.
Tip
Most acquirers don’t use an independent quality assurance function to objectively evaluate process and product quality. Therefore, using objective criteria to evaluate quality and allowing for escalation of issues without retribution becomes even more critical.
Tip
For less mature acquisition organizations, it may be best to use personnel outside the project team, perhaps from another acquisition project, to help perform more formal audits of critical processes or work products.
Traditionally, a quality assurance group that is independent of the project provides objectivity. However, another approach may be appropriate in some organizations to implement the process and product quality assurance role without that kind of independence.
If quality assurance is embedded in the process, several issues must be addressed to ensure objectivity. Everyone performing quality assurance activities should be trained. Those performing quality assurance activities for a work product should be separate from those directly involved in developing or maintaining the work product. An independent reporting channel to the appropriate level of organizational management must be available so that noncompliance issues can be escalated as necessary.
Tip
Because most acquisition project teams are very small, team members tend to have multiple roles and quality assurance activities are embedded within the process; therefore, it is important to ensure that objectivity is achieved..
Tip
Acquisition project leadership must create an environment in which team members are encouraged to identify and fix process and product defects.
Quality assurance should begin in the early phases of a project to establish plans, processes, standards, and procedures that will add value to the project and satisfy the requirements of the project and organizational policies. Those performing quality assurance participate in establishing plans, processes, standards, and procedures to ensure that they fit project needs and that they will be usable for performing quality assurance evaluations. In addition, processes and associated work products to be evaluated during the project are designated. This designation may be based on sampling or on objective criteria that are consistent with organizational policies, project requirements, and needs.
Hint
Establish your quality assurance approach early in the project and continually focus on identifying constraints that may keep you from achieving project objectives.
Use a risk-based approach when designating which processes and work products to evaluate.
When noncompliance issues are identified, they are first addressed in the project and resolved there if possible. Noncompliance issues that cannot be resolved in the project are escalated to an appropriate level of management for resolution.
This process area applies primarily to evaluations of project activities and work products, but it also applies to other activities and work products, such as training organizational support groups. For these activities and work products, the term project should be appropriately interpreted.
Tip
Limit the appraisal scope to the supplier processes that are most critical to the current phase of the project and use Class B or Class C (nonbench-marking) methods to help identify and mitigate project risk.
It also applies to the reviews of supplier process quality results as defined in the supplier agreement. For example, the supplier agreement can require the supplier to provide detailed appraisal results of mandatory, acquirer-scoped CMMI for Development appraisals of supplier processes.
X-Ref
For more information about interpreting CMMI achievements made by your suppliers, see “Understanding and Leveraging a Supplier’s CMMI Efforts: A Guidebook for Acquirers” at www.sei.cmu.edu/publications/documents/07.reports/07tr004.html.
Refer to the Solicitation and Supplier Agreement Development process area for more information about specifying evaluation of selected supplier processes and work products.
Refer to the Agreement Management process area for more information about managing conformance to supplier agreements.
Refer to the Acquisition Verification process area for more information about the verification of acquirer work products.
Refer to the Acquisition Technical Management process area for more information about evaluating the technical solution.
Specific Goal and Practice Summary
SG 1 Objectively Evaluate Processes and Work Products
SP 1.1 Objectively Evaluate Processes
SP 1.2 Objectively Evaluate Work Products and Services
SG 2 Provide Objective Insight
SP 2.1 Communicate and Ensure the Resolution of Noncompliance Issues
SP 2.2 Establish Records
Adherence of the performed process and associated work products and services to applicable process descriptions, standards, and procedures is objectively evaluated.
Hint
You will be able to determine the effectiveness of your processes and work products by evaluating adherence to objective criteria.
Regularly ask your customer and supplier how they perceive the effectiveness and efficiency of your processes and the quality of your work products contributing to their success.
Objectively evaluate designated performed processes against applicable process descriptions, standards, and procedures.
Objectivity in quality assurance evaluations is critical to the success of the project. A description of the quality assurance reporting chain and how it ensures objectivity should be defined.
The description of the quality assurance reporting chain is extended to include the relationship between the acquirer and suppliers. It is important to ensure that acquirer and supplier processes comply with applicable statutory and regulatory requirements.
The acquirer evaluates the project’s execution of acquirer processes, including interactions with suppliers, and reviews evaluation reports provided by suppliers to determine if they follow their processes. There should be sufficient process quality assurance to detect noncompliance issues as early as possible that may affect the acquirer’s or supplier’s ability to successfully deliver products to the customer.
Tip
Some of the most critical processes to evaluate are those that cross organizational boundaries. Acquirer-to-customer, acquirer-to-supplier, and acquirer-to-acquirer processes in a system of systems acquisition are good candidates to evaluate regularly.
Through the supplier agreement, the acquirer should retain the right to audit supplier processes if there is an indication that suppliers are not following acceptable processes.
Typical Work Products
- Evaluation reports
- Noncompliance reports
- Corrective actions
Typical Supplier Deliverables
- Reports resulting from evaluations carried out by the supplier
- Noncompliance reports
- Corrective actions
Tip
Quality is everyone’s job. It is important that everyone in the organization be comfortable identifying and openly discussing quality concerns.
Subpractices
- Promote an environment (created as part of project management) that encourages employee participation in identifying and reporting quality issues.
- Establish and maintain clearly stated criteria for evaluations.
Hint
The results of your quality assurance activities can identify best practices and processes that you should share with others in your organization. When working with multiple suppliers on the same project, best practices discovered through audit activities at one supplier location can become project-wide candidates for adoption to reduce project risk.
- Use the stated criteria to evaluate performed processes for adherence to process descriptions, standards, and procedures.
The supplier regularly provides process evaluation reports as defined in the supplier agreement.
- Identify each noncompliance found during the evaluation.
Analyze results of monitoring selected acquirer processes to detect issues as early as possible that may affect the supplier’s ability to satisfy requirements of its agreement.
- Identify lessons learned that could improve processes for future products and services.
Objectively evaluate designated work products and services against applicable process descriptions, standards, and procedures.
Hint
Resist the temptation to put total attention on the suppliers’ work products and not give “due diligence” to evaluating your own (acquirer) work products. Both must be of high quality to ensure program success.
You can embed objective evaluations of work products within some of the verification activities—particularly, peer reviews—though doing so requires care. (See the introductory notes for more information.)
In addition to objectively evaluating critical acquirer work products, the acquirer uses objective acceptance criteria to evaluate supplier deliverables throughout the project lifecycle. The acquirer’s acceptance criteria for supplier deliverables are consistent with project objectives and sufficient to allow the supplier to satisfactorily demonstrate that the product conforms to contractual requirements.
Refer to the Acquisition Verification process area for more information about verifying selected work products.
Typical Work Products
- Evaluation reports
- Noncompliance reports
- Corrective actions
Hint
Subpractices 4 through 6 recommend evaluation of work products at different times and from different perspectives. The important point is that you think broadly about what will give you the best objective insight you need during your project.
Subpractices
- Select work products to be evaluated based on documented sampling criteria if sampling is used.
- Establish and maintain clearly stated criteria for the evaluation of work products.
- Use the stated criteria during evaluations of work products.
- Evaluate work products before they are delivered to the customer.
- Evaluate work products at selected milestones in their development.
- Perform in-progress or incremental evaluations of work products and services against process descriptions, standards, and procedures.
- Identify each case of noncompliance found during evaluations.
- Identify lessons learned that could improve processes for future products and services.
Noncompliance issues are objectively tracked and communicated, and resolution is ensured.
Communicate quality issues and ensure the resolution of noncompliance issues with the staff and managers.
Noncompliance issues are problems identified in evaluations that reflect a lack of adherence to applicable standards, process descriptions, or procedures. The status of noncompliance issues provides an indication of quality trends. Quality issues include noncompliance issues and trend analysis results.
Hint
Noncompliance issues may be common in low-maturity organizations and should be addressed at the lowest possible level. Don’t “criminalize” those responsible for noncompliance.
When noncompliance issues cannot be resolved in the project, use established escalation mechanisms to ensure that the appropriate level of management can resolve the issue. Track noncompliance issues to resolution.
Noncompliance issues of both the acquirer and supplier are tracked and resolved.
Hint
Focus on fixing the process, not the people. Create an environment where everyone feels empowered to identify and correct errors without fear of retaliation.
Typical Work Products
- Corrective action reports
- Evaluation reports
- Quality trends
- Acquirer feedback to suppliers
Typical Supplier Deliverables
- Corrective actions
Subpractices
- Resolve each noncompliance with the appropriate members of the staff where possible.
The acquirer involves suppliers when resolving noncompliance issues, as appropriate.
- Document noncompliance issues when they cannot be resolved in the project.
- Escalate noncompliance issues that cannot be resolved in the project to the appropriate level of management designated to receive and act on noncompliance issues.
- Analyze noncompliance issues to see if there are quality trends that can be identified and addressed.
- Ensure that relevant stakeholders are aware of results of evaluations and quality trends in a timely manner.
Hint
Quality assurance activities are often seen as nonvalue-added. The status and results of quality assurance activities should be reported regularly. Also, consider reporting a “compliance percentage” rather than the number of noncompliance issues. This may encourage friendly competition across projects and increase appreciation for the quality assurance role.
- Periodically review open noncompliance issues and trends with the manager designated to receive and act on noncompliance issues.
- Track noncompliance issues to resolution.
Establish and maintain records of quality assurance activities.
Typical Work Products
- Evaluation logs
- Quality assurance reports
- Status reports of corrective actions
- Reports of quality trends
Tip
Records provide a way to identify trends in quality assurance activities (including noncompliance issues) that allow the organization to identify where additional guidance or process changes are needed.
Subpractices
- Record process and product quality assurance activities in sufficient detail so that status and results are known.
- Revise the status and history of quality assurance activities as necessary.