After completing this chapter, you will be able to:

• – understand the importance, as well as the effect, of including SQA in projects that involve external suppliers;
• – know the requirements of the ISO 9001, ISO/IEC/IEEE 12207 standard, and the CMMI^® model for the management of agreements with suppliers;
• – recognize the difference between suppliers and external participants;
• – communicate and manage the risks associated with external participants;
• – be aware of the two main software contract reviews;
• – understand the requirements of the IEEE 730 standard regarding the monitoring of suppliers in the quality assurance plan of the project.

12.1 Introduction

When software work involves external suppliers, the software quality assurance (SQA) staff and project managers should be knowledgeable in the management of suppliers and agreements/contracts. The quality of a relationship between partners is a complex concept and it is key to the success of the project. We believe that adequate preparation, the choice of an adequate agreement or contract type, frequent reviews, and follow-up are fundamental for a good relationship. The development of contractual clauses that apply the knowledge described in this book is also key for delivering quality software in this complex situation.

Ensuring quality results in this type of project requires that the supplier's personnel are involved and knowledgeable regarding SQA processes. To ensure this, we expect that the supplier provides a SQA plan (SQAP) (in addition or included in the project and technical plans) before we can finalize contractual negotiations. This allows SQA, as well as client's project manager, to assess the SQAP intentions of the external supplier for this project. Thus, the acquirer SQA function has the important and arduous task to ensure early discussion on this type of software project.

We have observed that customer satisfaction will be higher when the supplier adopts a collaborative strategy as well as simple and straightforward language. The following sections present an overview of the required knowledge in the management of suppliers and contracts.

12.2 Supplier Requirements of ISO 9001

Clause 8.4 of the ISO 9001 standard describes the control of externally provided processes, products, and services, whether through:

• – purchasing through a supplier;
• – an arrangement with an associate company; or
• – outsourcing to an external provider.

The controls required for external provision can vary widely depending on the nature of the services or products acquired. The organization can apply risk-based thinking (that was covered in a previous chapter) to determine the type and extent of controls appropriate to particular suppliers and externally provided services and products.

Many topics presented in ISO 9001 are helpful in ensuring that a supplier delivers quality software. For example, clause 4.4 insists that processes needed (i.e., inputs and outputs), their interactions, and responsibilities and authorities be clear to ensure that the quality system works effectively. Clause 5.1 goes further and recommends that the supplier demonstrate leadership and commitment. Clause 6.2 discusses the establishment of quality objectives and the planning to achieve them. We have already covered this topic in an earlier chapter.

Clause 8.4 of ISO 9001 describes the controls required to manage a supplier's processes, products, and services: “The organization shall ensure that externally provided processes, products and services conform to requirements.

The organization shall determine the controls to be applied to externally provided processes, products, and services when:

• – products and services from suppliers are intended for incorporation into the organization's own products and services;
• – products and services are provided directly to the customer(s) by suppliers on behalf of the organization;
• – a process, or part of a process, is provided by a supplier as a result of a decision by the organization.

The organization shall determine and apply criteria for the evaluation, selection, monitoring of performance, and re-evaluation of suppliers, based on their ability to provide processes or products and services in accordance with requirements. The organization shall retain documented information of these activities and any necessary actions arising from the evaluations.” [ISO 15]

Additionally, clause 8.5 of ISO 9001 specifies the responsibilities of the procuring organization regarding intellectual property [ISO 15]: “The organization shall respect the property of customers or external service providers when it is under their control or being used. The organization shall identify, verify, protect, and safeguard the property that customers or external service providers have provided them for use or incorporation into their products and services.”

12.3 Agreement Processes of ISO 12207

There are two agreement processes in ISO 12207: the acquisition process and the supply process: “These processes define the activities necessary to establish an agreement between two organizations. If the acquisition process is invoked, it provides the means for conducting business with a supplier. This may include products that are supplied for use as an operational software system, services in support of operational activities, software elements of a system, or elements of a software system being provided by a supplier. If the supply process is invoked, it provides the means for an agreement in which the result is a product or service that is provided to the acquirer.” [ISO 17].

According to ISO 12207, the purpose of the acquisition process is to obtain a product or service in accordance with acquirer's requirements. The process begins with the identification of the customer needs and ends with the acceptance of the product and/or service needed by the acquirer. The following activities are described in the standard [ISO 17]:

1. prepare for the acquisition: the acquirer defines a strategy, describes what is to be acquired in enough detail (i.e., system/software requirements) so that the supplier can understand it;
2. advertise the acquisition and select the supplier: the acquirer communicates the request for the supply of a product and/or services, then using an established procedure, the acquirer evaluates and selects one or more suppliers;
3. establish and maintain an agreement: the acquirer prepares and negotiates an agreement with the supplier. During this activity, he identifies the necessary changes and their impact on the agreement with the supplier including: acquisition requirements, costs and schedule, and many other topics like acceptance criteria, warranty, and licensing;
4. monitor the agreement: the acquirer assesses the execution of the supplier's activities in accordance with the software review and audit processes (as seen in earlier chapters). He also provides data needed by the supplier and resolves issues in a timely manner;
5. accept the product or service: the acquirer conducts the acceptance review and testing of the deliverable. During this process the acquirer confirms that the delivered product or service complies with the agreement, provides payment or other agreed consideration, takes on the ownership of the configuration management (as discussed in a previous chapter) and finally closes the agreement.

According to the ISO 12207, the purpose of the supply process is to provide an acquirer with a product or service that meets the agreed requirements. The following activities are described in the standard [ISO 17]:

1. prepare for the supply: the supplier determines the existence and identity of an acquirer who has a need for a product or service. Once done he defines a supply strategy;
2. respond to a request for supply of products or services: the supplier evaluates a request to determine its feasibility and how to respond. Then he prepares a response that satisfies this solicitation;
3. establish and maintain an agreement: the supplier negotiates an agreement with the acquirer that includes acceptance criteria. The supplier may identify necessary changes to the agreement and their impact as part of a change control mechanism. Then a negotiation will take place and the final agreement is made official;
4. execute the agreement: the supplier executes the agreement according to the established project plans. The most important agreement execution activities are summarized as: reviews, choosing an appropriate software life cycle model, detailing project management plans, including an SQAP, performing V&V, assessing the execution and quality and managing subcontractors;
5. deliver and support the product or service: the supplier delivers the software product or service as specified in the agreement. He also needs to provide assistance to the acquirer in support of the delivered software.

A software quality management process suited for the project is established. This is the mechanism to assure quality and conformance to established plans. As many software engineering processes work together, the SQA process should be coordinated with the software V&V, review and the audit processes. It also requires that a SQAP (software quality assurance plan) be developed and typically include the following:

• – quality standards, methodologies, procedures, and tools for performing the SQA activities;
• – procedures for contract review and coordination thereof;
• – procedures for identification, collection, filing, maintenance, and disposition of quality records;
• – resources, schedule, and responsibilities for conducting the SQA activities;

Once a plan is in place, it will be necessary to schedule the SQA activities and execute the plan. A problem resolution process will be used between the acquirer and the supplier during this period to solve all outstanding issues. For this process to work correctly, individuals performing SQA functions should have a position within the organization that provides an unimpeded communication mechanism with management. This will allow free circulation of the information for problem resolution. Before delivery, software products will be verified to ensure they have fully satisfied their contractual requirements and are acceptable to the acquirer.

12.4 Supplier Agreement Management According to the CMMI

We know that the CMMI^® for Development (CMMI-DEV) is a descriptive model in that it describes the essential attributes (or major attributes) that are expected from the processes to be executed in an organization that is working, in this case, in the Supplier Agreement Management process area of maturity level 2 of the staged representation. This model is also used as a normative model because the objectives and practices describe the practices that the acquirer expects from the in-house or external suppliers undertaking projects in a contractual context. CMMI proposes that project managers, in the acquirer's organization, must master the agreement management process with suppliers.

Agreement management involving external suppliers is set at level 2 of the CMMI-DEV. This is mainly due to the fact that acquiring software products and services is much more common nowadays. CMMI describes the specific goals (SG) and specific practices (SP) required as [SEI 10a]:

SG 1 Establish supplier agreements:

• – SP 1.1: Determine acquisition type
• – SP 1.2: Select suppliers
• – SP 1.3: Establish supplier agreements

SG 2 Satisfy supplier agreements:

• – SP 2.1: Execute the supplier agreement
• – SP 2.2: Accept the acquired product
• – SP 2.3: Ensure transition of products

Figure 12.1 shows the interaction between the SP of this process area.

In addition, according to the CMMI, the organization should commit that its software projects follow a written policy for the management of software acquisition. Furthermore, a contract manager should be assigned for the establishment and management of contract activities. CMMI also requires that these contract management activities are reviewed with the project manager on a periodic basis rather than on an event-driven basis.

In addition to activities directly carried out regarding the project, CMMI suggests that the organization has implemented project verification procedures. It is therefore expected that the contract management activities be reviewed with management on a periodic basis. To support this activity, SQA and project managers should carefully review the activities and work products described in the agreement and/or perform third-party audits, as described in an earlier chapter.

The following anecdote describes how a public service manages its suppliers using the CMMI model.

12.5 Managing Suppliers

As we have seen, the need to manage suppliers during a software project is presented in many standards. The number of external providers that contribute to a software project can be important and are often working in the background. There can also be a partnership between suppliers. The bigger the project, the more complex this situation can become. The types of suppliers can be:

• – subcontractors: who take responsibility for a specific part of a contract. We will call on contractors when seeking specific software expertise and also to ensure the timely availability of experts when needed;
• – software package suppliers: offering off-the-shelf software ready to be adapted and implemented. These external providers are becoming more popular because they offer proven software packages that can reduce new software development costs and delays;
• – consultants: who are hired on the project to help with specific tasks, such as explain the requirements and business rules, develop interfaces with other internal software, evolution and maintenance staff, and IT infrastructure staff. These additional resources, coming from outside the organization, bring considerable expertise to the project's success but must be coordinated properly.

As the number of people involved grows, the more coordination and quality issues can arise:

• – deadlines: the orchestration of the various stakeholders may become more difficult and many meetings are needed to address the issues, verify each intermediary deliverable and coordinate work. A problem with a key supplier will have a direct effect on the project;
• – quality of deliverables: we saw that quality issues can be varied, for example: defects, failure to comply with established rules, incomplete intermediate deliverables and misunderstanding of a requirement. These problems will create rework and additional testing effort to re-verify deliverables;
• – transition difficulty: during the transition, the software operation becomes the responsibility of the customer. Customer software maintainers and IT operations can also identify non-conformities and ask the external providers to rework their deliverables before accepting them. It is therefore important that the SQA function help project managers that are involved with several external providers in understanding the complex process that a project will require. It can support this by describing the SQA tools available to help:

• prevent delays and make sure to prevent the arrival of impending problems;
• ensure early assessment of the quality of deliverables (preventive approach);
• pay attention to potential downstream requirements of IT operations and software maintenance/support requirements;
• keep an eye on the performance of each stakeholder involved.

A solid SQAP (discussed in the next chapter) as well as a clear process map of the software acquisition process are two key elements of success for these complex software projects. The quality plan will bring more precision to the obligations of each of the stakeholders and the process mapping will clarify the roles, activities, and deliverables of the project from beginning to end.

12.6 Software Acquisition Life Cycle

Software projects that involve suppliers are often more complex than traditional software development projects. To illustrate this, we will describe an acquisition process specially designed for the acquisition of off-the-shelf products like SAP/R3 solutions, Oracle Financials and similar software packages. There are several factors to consider before purchasing such software solutions. The software acquisition life cycle description will define and clarify the activities and the roles involved in the software procurement process. It is an essential part of SQA for this particular situation.

A supplier management (SM) process requires goals, applicability, objectives, and a detailed process mapping of the activities. The example that follows shows a high level vendor management process for a real company.

The software acquisition life cycle is complex and needs to be described in more detail so that external suppliers clearly understand what is expected of them. For a large Canadian equipment distributor, three detailed process maps were developed to better explain the detailed activities to the supplier. These maps describe the following three steps (see Figure 12.2):

• – the request for information (RFI) stage (stage 1);
• – the supplier selection stage (stage 2);
• – the adaptation and implementation of the software package (stage 3).

The RFI stage represented here aims at identifying and selecting the best candidates to lead a complex software project. The first activity of this process is to document business requirements and existing processes/systems. The result of this activity is high-level requirements. This is used as an entry point to develop the RFI, which, once approved by the legal department, will be sent out to potential suppliers. Each supplier response will then be evaluated and potentially used to help develop a request for proposal (RFP).

As we can see from Figure 12.2, a number of roles and responsibilities as well as templates are described and greatly clarify all the activities. When the customer takes the time to provide this level of information, the quality of the project increases. The two other process maps of this process (i.e., stage 2 and stage 3) are available on this book's website. The next section presents software contract types that can be considered for software suppliers.

12.7 Software Contract Types

We have seen that the software acquisition life cycle processes require choosing a contract type. Contracts, in the software domain are complex and involve legal, project management, and technology-related clauses. There are many types of contracts. The simplest types involve consultancy and contract hire where expertise is needed: for example, to document requirements, to write specifications, or convert data. They are relatively simple because the sums of money involved are small and when the work is finished, the deliverables are verified by the customers. For larger endeavors where the customer would like a turnkey solution, this contract type is not fit for that purpose.

Another type of contract is the fixed-cost contract. When should this type of contract be used? Projects with clear tasks are good candidates for a fixed price contract. Some organizations are forced to use this type of contract by law. To clarify deliverables, during requirements generation, a column can be added to the WBS definition of the tasks identifying each status task as clear/not risky or unclear/risky/surge. For example, consider whether the daily operations and maintenance of a system, as well as regularly scheduled maintenance requirements, are clearly identified and their costs reasonably estimated. An unclear/surge task, like unscheduled maintenance or repair, could be contracted separately as time and materials, labor hours plus other fixed costs.

Four popular contract types that are commonly used for software services acquisition are listed below. Each type offers a different risk profile for the external provider and the customer:

• – fixed priced contract;
• – cost plus percentage of cost;
• – cost plus fixed fee;
• – risk sharing.

For all these types of contracts, it is necessary that SQA takes the time to propose appropriate contract clauses to the project manager. This will ensure the quality of results (i.e., ensure alignment of the contract clauses with the process, project plan, and SQAP). In addition, it is important to describe, among other things, how cost overruns will be handled as well as the supplier cost control process.

12.7.4 Risk Sharing

This type of contract is well adapted for complex software acquisitions. With this type of contract, the supplier is reimbursed for allowable expenses for the execution of the contract. In addition, the supplier has the opportunity to receive a bonus if the work is completed early. This bonus will be paid if the final cost is less than the agreed upon estimated price. The savings will be shared between the supplier and the customer. Both the supplier and the customer share the advantage of completing the project ahead of the deadline. On the other hand, missing the deadline is also a shared risk.

Here is an example of the use of a risk sharing approach used as part of a large financial software replacement project. In Figure 12.3, the costs borne by the customer are in black and the costs borne by the integrator are shown in white. The supplier agreed to implement this software package for $1,174,902 within 42 working days (i.e., 8 hours per day). This estimate includes some uncertainty and both partners are willing to share the risk associated with missing this deadline but the customer wishes to obtain a guarantee of a fixed budget ceiling in the contract. Given that the supplier that was chosen has a very good track record of executing this type of project, this estimate is solid.

The stakeholders (i.e., the customer and supplier) agree that after a 5% overrun, the costs will be shared in this way—60% by the supplier and 40% by the customer. In addition, the customer wishes to establish a limit (a guarantee) because a fixed budget needs to be approved. This limit is set at 25% of overrun. This means that beyond a 25% overrun of the estimated budget, the supplier will have to assume all the extra costs until the delivery. A careful and professional supplier will therefore prepare his proposal accordingly and have confidence to assume that risk.

Let us look at the details of this contract. Table 12.1 describes the change in percentages and their effects. To calculate the values in this table, a simple slope formula is used: for example, for the supplier y = 4.7x − 146.8, where x is the number of days to complete the project and y is the percentage of cost over budget. This formula is derived by considering the values from two data points in Figure 12.3: y₂ = 100%, y₁ = 60%, x₂ = 52.5 days, and x₁ = 44 days. For example, if the project exceeds 14% from the target date, the integrator will assume y = (4.7 × 48) − 146.8 = 78.8% of cost overrun. Alternatively, the customer will only pay the remaining 21.2%. The interesting feature of this type of contract is that there is a maximum amount of risk (i.e., cost) that can be identified at 12% = $36,226.

Price of contract (set by RFQ)			$1174902
Estimated effort (set by RFQ)			42 days		Slope	4,7
Daily cost			$27974		Intercept	−146,8
% late (days)	Day	% risk of supplier	% risk of customer	Late ($)	Cost to supplier	Cost to client
On time	42	0	0	$0	$0	$0
2%	43	0	100	$27974	$0	$27974
5%	44	60	40	$55948	$33569	$22379
7%	45	65	35	$83922	$54297	$29624
10%	46	69	31	$111895	$77655	$34240
12%	47	74	26	$139869	$103643	$36226
14%	48	79	21	$167843	$132260	$35583
17%	49	84	17	$195817	$163507	$32310
19%	50	88	12	$223791	$197384	$26407
21%	51	93	7	$251765	$233889	$17875
24%	52	98	2	$279739	$273025	$6714
25%	52,5	100	0	$293726	$293726	$0
Client budget for worst case = 1174902+$36226 =				$1211128

We know that schedule overruns are unfortunately very likely with these types of software projects. Since all cost overruns beyond 25% will be borne by the supplier, it is possible to calculate a maximum guaranteed budget for the customer should things go very wrong. Looking at the cost to the client, the maximum value is reached at a 12% overrun. With this type of contract, the customer's project manager is fully confident that with a budget of $1,211,128, he incurs no additional risk of overruns. With this assurance, the project manager and SQA team can focus on the quality of the deliverables.

To advise project teams and assist in the establishment of well-designed contracts, the SQA specialist needs to be familiar with software contract types and clauses. After planning a contract strategy and before signing this agreement, contract reviews are required to ensure the alignment of the project plan, quality plan and contract clauses. The next section presents the recommended contract review activities.

12.8 Software Contract Reviews

There are two major contract reviews mandated to ensure the quality of a software agreement. These two reviews (initial and final) aim to improve the odds of meeting budget, schedule, and target quality. The requirement for a contract review process should originate from the customer. Suppliers should make a substantial contribution to these activities.

Contract review objectives are:

• – to identify the factors that influence the magnitude of each review;
• – identify the difficulties in conducting the contract review;
• – to explain the activities and objectives for the implementation of each contract review;
• – to discuss the importance of conducting these reviews.

12.9 Supplier and Acquirer Relationship and the Sqap

IEEE 730 [IEE 14] has been created for the situation where software development is considered to be carried out by a supplier and delivered to an acquirer. For this reason, the entire standard is pertinent when a supplier and an acquirer enter into a relationship for a software acquisition. This complex software project needs to describe, in its SQAP, how suppliers will be managed to ensure a quality delivery. This standard requires that means to achieve quality be described in a plan and a contract to ensure that both supplier and acquirer clearly understand the requirements as well as their responsibilities.

IEEE 730 explains, in detail, every SQA activity presented in the SQA process of the ISO 12207 standard.

12.10 Success Factors

We summarize the factors that affect quality during the software acquisition process in the next text box.