Acronym

26.1 Introduction

“Sourcing refers to the act of transferring work, responsibilities, and decision rights to someone else” [1]. Outsourcing is the act of sourcing work to an outside organization; that is, to an external entity. Outsourcing software development and verification is not new, but it is definitely more prevalent now than in the past. There are many aspects to consider when outsourcing; however, it is not feasible to examine them all here. This chapter concentrates on outsourcing some or all of the safety-critical software development and verification to one or more teams. This chapter investigates why outsourcing is promoted, potential risks and challenges of outsourcing and offshoring, and recommendations for addressing the risks.

This chapter concentrates on the topic of outsourcing from a safety and certification perspective. It is primarily written for software technical leaders who make decisions about outsourcing and manage outsourced resources for safety-critical software; however, it is also applicable to developers, verifiers, quality assurance, and certification personnel. Many programmatic aspects are discussed because they impact the software life cycle and the end product. Political, legal, and contractual aspects of outsourcing are not examined. Please note that this chapter considers outsourcing from a United States (U.S.) perspective. The concepts should apply if you live elsewhere; however, there are some specific concerns for U.S.-based companies to consider when offshoring civil aviation products.

From my avionics experience, outsourcing tends to fall into three categories: (1) products (such as line replaceable units or entire systems), (2) software or electronic hardware applications, or (3) labor services (e.g., portions of software development and/or verification). From a U.S. perspective there are three locations where outsourcing occurs: (1) within the U.S., (2) outside of the U.S. in countries with bilateral agreements with the Federal Aviation Administration (FAA), and (3) outside of the U.S. in countries without bilateral agreements with the FAA. The location is important for several reasons, including experience, domain expertise, legal framework, and certification requirements. The FAA, in particular, considers the location of the outsourced organization because it directly affects who is responsible for the certification oversight. Projects performed in the U.S. are overseen by the FAA. For activities performed in countries with bilateral agreements with the FAA, the FAA may request the international certification authority to perform the oversight.^* In countries without bilateral agreements with the FAA, there is no local certification authority for the FAA to legally approve or accept; therefore, it could lead to additional resources for the FAA to oversee such activities. This can be considered an undue burden to the U.S. government, since the FAA is not a fee-for-service organization. If the outsourcing effort is classified as an undue burden, the resulting work may not be accepted by the FAA.

Experience shows that software activities performed offshore require careful consideration. Therefore, the certification authorities consider offshore projects to be higher risk. The FAA doesn’t distinguish between outsourced, offshored, and subcontracted work—entities that perform such work are simply referred to as suppliers. Even companies that own off-site facilities inside or outside the U.S. need to address the sourcing issues.

For the rest of this chapter, the term outsourcing will include subcontracted or subsidized work performed onshore (within your country), near-shore (in a country that shares borders with your country), or offshore (in a country that does not share borders with your country). These terms are independent of who owns the organization performing the work. Some argue vehemently that work done by an owned subsidiary is not considered outsourcing. I agree that the business arrangement is different and that the risk should be lower; however, many of the issues are similar. Therefore, I include them in the outsourcing category. From a certification perspective, the business arrangement isn’t as important as the quality of the work. History shows us that many offshored projects have quality issues, some of which are described later. As companies learn to better address the quality challenges, this perception may change.

26.2 Reasons for Outsourcing

There are numerous reasons for outsourcing; some are economic and some are technical. A list of the common reasons is noted in the following:

• It helps address engineering shortage. There is a shortage of software engineers in the western world (e.g., U.S., Canada, and Europe). To meet project demands, outsourced engineers are used to augment a company’s existing team and to help with project sequencing. This allows a company to take on additional work and to allocate internal resources/staff to the most critical projects.

• The labor costs are lower in some countries. The decreased labor costs are also appealing to business managers. However, in my experience, the cost tends to be proportional to the quality of work produced. As my dad is so fond of saying: “You get what you pay for.” Due to inexperience, inefficiencies, and poor oversight, I’ve seen a cheap engineer end up costing over three times more than an experienced high-rate engineer. When looking at labor rates, business managers frequently forget that outsourced teams require oversight; the oversight ratio of in-country engineers to offshore engineers ranges from 1:1 to 1:30, depending on the nature of the work being outsourced and the team performing the work. Once an offshore team becomes more experienced, their labor rates tend to rise, so they are no longer cheap. Managers who think they will save a bundle with lower outsourced labor rates should be forewarned: there are no free lunches in safety-critical software development.

• Some companies have specialized skills. Rather than hiring or developing expertise in a new or noncore area (e.g., operating systems), it may be more practical to outsource the work to an organization that specializes in that area. This allows a company access to expertise, knowledge, and capabilities outside its own boundaries [1].

• It allows around-the-clock work. By outsourcing to teams in significantly different time zones, it can allow for around-the-clock productivity. There are advantages and disadvantages to this. An around-the-clock schedule means many late-night and early-morning meetings or teleconferences, which can burn out the team. Running an aroundthe-clock operation also takes a global infrastructure that needs to be developed in advance of the project launch.

• It is mandated. Some outsourcing (particularly to offshore countries) is mandated by upper-level management for other business considerations or by aircraft contracts. The mandates are usually accompanied with a certain percentage or amount of work that must be outsourced to certain countries or organizations. Unfortunately, these are often the projects that struggle the most to effectively outsource.

When done properly, outsourcing can be a win–win situation for everyone involved. However, I’ve seen enough outsourcing fiascos (both within and outside the U.S.) to know that it requires considerable tender loving care and constant oversight to make it successful.

There is no magic formula or a one-size-fits-all solution for outsourcing. The remainder of this chapter documents some of the cautions and suggestions for how to tackle the issues.

26.3 Challenges and Risks in Outsourcing

Outsourcing endeavors vary significantly—all the way from a few tasks outsourced to a domestic supplier, to the entire system outsourced to an offshore entity in a country without bilateral agreements, with a lot of variants between these two extremes. This section examines some of the common challenges. Recommendations to mitigate these challenges are discussed in the next section. The first nine challenges are relevant to domestic and offshore teams. The last eight items are more applicable to offshore work.

Please note that throughout this section, the outsourced team is called the supplier, regardless of the type of work performed or the nature of the business relationship. The supplier is the organization selected to do some or all of the work for a safety-critical software project. Also, the company that utilizes the outsourcing team (the supplier) is called the customer throughout this section. The customer is assumed to be ultimately responsible for the DO-178C compliance.

Challenge 1: Lack of experience. If the supplier doesn’t have the appropriate software engineering, management, certification, and safety experience, success is unlikely. Years ago in my late 30s, I visited an offshore supplier for a level A project. I felt like a grandmother. Nearly every team member was right out of college. I respect youth, but it needs to be balanced with and guided by experience. There’s a common saying that goes: “If you want to win, you don’t enter mules in the Kentucky Derby.” To win the Kentucky Derby, you need a thoroughbred. Likewise, talented and experienced engineers are needed to develop and verify high quality software.

Challenge 2: Lack of domain knowledge. If an engineer lacks experience in a domain, the quality of the work suffers. While the engineer may be incredibly bright, he or she may not have the knowledge to make informed decisions. Outsourcing safety-critical embedded software development and verification to a company that has telecommunication experience is probably not a good fit for most aviation software, since such engineers generally lack an understanding of real-time, deterministic, safety-driven software, as well as the discipline and rigor required by DO-178C.

I recently consulted on a level A project in which the software lead was doing his first avionics and certification project. He had worked on farm equipment in the past. He was a bright engineer; however, he didn’t understand flight controls, safety, DO-178B objectives, or certification expectations. Despite some people’s thinking, avionics require domain experience, a commitment to excellence, and a perfectionist mentality. Engineers cannot be moved around like pieces on a game board, if quality is the desired outcome.

Challenge 3: Unstable workforce. High staff turnover can be an issue for some suppliers. Turnover may be due to low morale, inadequate pay, lack of reward opportunities, the country’s growth dynamics, etc. It takes years to develop a skilled engineering team with the right people in the right positions. If the staff continually churns, retraining and starting at the bottom (or close to it) of the learning curve are constant. This results in lower quality work.

Challenge 4: Unrealistic expectations. A number of efforts fail because of unrealistic expectations. Many projects use what I call success-based planning. This occurs when project managers don’t plan for the learning curves, transition times, probable performance rates, failed tests, audit findings, or schedule dependencies that invariably happen. As a result, projects are set up for failure. Expecting high quality software with a short schedule from an untested team is not realistic. To put it another way, expecting too much too fast is foolish. Schedule pressures often lead to bad decisions (e.g., throwing more engineers at the problem rather than letting the right and qualified engineers do their job successfully). Likewise, it’s unrealistic to expect an inexperienced team to produce quality products (steep learning curves aren’t always possible to climb). Good quality requires adequate resources, a reasonable schedule, and experienced staff.

Challenge 5: Rapid growth. Success can lead to disaster. Some suppliers grow too fast. In an effort to meet the demands of multiple customers or projects, they may spread themselves too thin, pull in less experienced people to help, or have experienced engineers jump between projects. It is a challenge to get the right people in the right place at the right time. One company I consult with is proposing doubling or tripling their workforce every year for the next 5 years, in a country with little safety-critical software experience. It does not seem like a reality-based plan to me.

Challenge 6: Overselling the company and its abilities. Some software suppliers have smooth-talking marketers but are unable to deliver the goods. I have attended several marketing pitches where the marketing representative threw around DO-178B or DO-178C and certification jargon like he knew what he was talking about. But once I started asking questions, it became obvious that the pitch was like a television advertisement. Buyers beware! Deceptive marketing isn’t just for used cars. Even conscientious suppliers can sometimes oversell their abilities.

Challenge 7: Changing processes. Some companies perform well when they follow their own process. However, when the processes change (perhaps due to the needs of the customer or domain), they may not adapt well. In my experience, the larger the team, the more difficult it is to change a process. It’s difficult to train everyone and enforce the change, especially if the team members don’t understand why the change is needed.

Challenge 8: Poor selection process. When determining what work will be done and where, it is important to have an objective assessment. Oftentimes, teams are forced to select certain suppliers (perhaps based on location or marketing relationships) rather than the best supplier. Failure to properly assess the skills, depth, and record of the supplier leads to problems down the road. I once witnessed a project that had three potential suppliers bid for the work. They chose the one who quoted the lowest estimate, even though there were clear signs that the low bidder would be unable to deliver the work with quality and on time. The endeavor ended up costing at least 10 times the original bid of the low bidder, which equated to about three times the bid of the more qualified supplier. And, incidentally, it was 2 years late.

Challenge 9: Poor quality. It is common to have challenges getting quality work from outsourced suppliers. This can be a challenge within a single company too, but it’s often more difficult to identify the quality issue in another organization until later in the process or life cycle (and possibly too late to change the direction). Quality is more than just tracking metrics. Without proper technical oversight, it could be late in the project before one realizes the requirements are bad, the design is flawed, and the code doesn’t work. Sometimes, even experienced companies can have a quality lapse; perhaps they get stretched thin, hire a new program manager that isn’t ready for the job, or succumb to the schedule pressure.

Challenge 10: Language issues. Communication with some offshoring suppliers can be a challenge. If a team doesn’t have a solid grasp of English,^* it’s difficult to write unambiguous requirements, accurately test the requirements, write useful problem reports, communicate issues, etc. I’ve consulted on a couple of projects where a translator was needed to communicate. This can be very time consuming and error-prone. Combine the language issues with inexperience and/or a lack of safety culture and the probability of success drops significantly. While on the topic of language, it should be noted that the FAA requires certification-related data to be in English and accessible in the U.S. [2].

Challenge 11: Cultural issues. It is important to understand cultural differences when outsourcing. There can be cultural differences even within the U.S., but the greater challenges are with international projects. Here are some examples of cultural aspects to be aware of:

• Different cultures are often motivated differently. What works in the U.S. to motivate a team may not work in Europe, Asia, or South America.

• In some countries it’s considered disrespectful to point out negative things about someone or something. However, when doing software verification, the goal is to discover errors. Without a proper verification mentality, one could end up with the most successful test program ever, but the system still doesn’t work.^*

• Not all countries have the same safety standards or value for human life. The evaluation of software defects requires a zero-defect mentality for safety-critical software. In one instance, an offshore company that I consulted with didn’t understand the need to test or fix errors in level A software.

• Some cultures may not respect or have a concept of excellence. In some cultures, it may even be considered wrong to have pride in your work. However, developing safety-critical software requires a perfectionist mentality and attention to detail. If the engineers are satisfied with mediocre or good enough work, it’ll be difficult to prove that it’s safe.

Every culture has its strengths and weaknesses. It’s important to understand the cultural background of all entities, provide cultural training for all team members involved, and allow time for diversity integration.

Challenge 12: Ethical issues. Not all cultures adhere to the same ethical standards. In some cultures it’s acceptable to lie to get ahead, to steal as long as you don’t get caught, and to offer bribes. Some countries are suspected of introducing security vulnerabilities and backdoors in the products they develop.

Challenge 13: Education issues. Not all engineering programs are created equally. In some schools across the world, 50% is considered passing. Just because someone has a degree in engineering doesn’t mean they can engineer. Be sure to check out the quality of education, as well as the experience, of the supplier’s engineers.

Challenge 14: Hidden expenses. There can be, and probably will be, unexpected expenses when offshoring. Business class airline tickets and safe hotels add up fast. Despite the advances in video conferencing and network-based meetings, there is still a need for in-person meetings and oversight. Additionally, setting up an infrastructure (equipment, high-speed network access, etc.) can be costly.

Challenge 15: Losing experienced engineers. Despite best efforts to put a positive spin on outsourcing and to encourage teamwork, sometimes experienced engineers are lost when outsourcing occurs. The customer’s engineers may see the outsourcing arrangement as giving away their job or they may want to design and code rather than oversee a supplier. Burnout due to early morning and late-night meetings can also be a factor.

Challenge 16: Certification challenges. As mentioned earlier, certification authorities have seen issues with outsourcing (particularly offshoring), so they normally require special justification for and oversight of offshore suppliers. If work is performed in a country without bilateral agreements, there needs to be justification to ensure there is no undue burden. Some suggestions for how to do this are examined in the next section.

Challenge 17: Post-project amnesia. I’ve consulted on projects that I considered to be certification fiascos. Yet, the mere fact that they made it to the end, without bankrupting the company, is somehow twisted to be viewed as a success. I’ve seen poor-performing outsourced companies rewarded with additional work because management seems to forget the prior chaos that took place. The supplier’s marketing department may even proclaim they were key to the project’s success (without mentioning the hassles they caused). Unless specific action is taken to correct the issues from the past, the issues will likely happen again. As I’ve heard it said: “The definition of insanity is doing the same thing again and expecting different results.” Unfortunately, when it comes to outsourcing, particularly offshoring, I’ve seen enough insanity to make me a skeptic.

26.4 Recommendations to Overcome the Challenges and Risks

Despite the challenges identified in the previous section, outsourcing can be successful. It can provide great benefits when done properly. This section presents some strategies and recommendations to handle the challenges of outsourcing. As noted earlier, the focus is on safety and certification. An organized process is essential to developing software that performs its intended function. Therefore, some of the practical programmatic practices that impact the overall software quality when outsourcing are covered.

Even as a youngster in rural America, I had a desire to see the world. Engineering has opened the doors of the world for me. I enjoy meeting, working with, and eating with people from all over the world. It is enlightening to interact with such diverse, interesting, and talented people across the globe. It takes some time to grow a project or a team to reap the benefits of outsourcing, but once you do it, it can be extremely fulfilling. These recommendations are intended to help you and your team experience such fulfillment.

As with the last section, the outsourced team is called the supplier and the company that utilizes the outsourcing team is called the customer throughout this section.

Recommendation 1: Determine readiness for outsourcing. Some companies outsource because they see everyone else doing it. Without the proper preparation and groundwork, outsourcing can be a dismal failure. Before diving into an outsourcing arrangement, it’s important to evaluate the customer’s outsourcing readiness. In his book, entitled Software Without Borders, Steve Mezak provides a 20-question outsourcing readiness test to evaluate an organization’s readiness for outsourcing. The test examines outsourcing experience, technology, business situation, and management approach [3].^* Such an evaluation can help identify if outsourcing is a wise decision and where additional help may be needed.

Recommendation 2: Ensure top-level management’s commitment. It is essential to have both the short-term and long-term support of executive-level management. Without this, failure may be declared at the first sign of trouble. Management must be provided a complete and accurate understanding of the needs, risks, options, and expectations. Both the customer and supplier teams should be careful not to promise what can’t be delivered. It’s okay to be optimistic, but the optimism ought to be balanced with an accurate reflection of reality. A well-informed and fully committed management can open doors, provide support, and give vision to the project and to the overall enterprise.

Recommendation 3: Use a systematic and organized approach. Some organizations jump into an outsourcing arrangement with both feet before they even have a plan—it’s the ready-fire-aim approach to software development. One doesn’t build a house without a blueprint and one cannot effectively outsource part, or even all, of the company’s work without an organized approach. The Outsourcing Handbook by Mark Powers et al. explains the need for a processdriven approach to outsourcing and proposes an outsourcing life cycle with

seven phases. The seven phases are cyclical (unless the exit option is exercised), and each phase is performed independent from the phase that follows it. It is important to address each phase well and not to rush through any one of the phases. Each phase is briefly discussed here [1]:

1. Strategic assessment. This entails making a business case to identify the benefits of an outsourcing arrangement (including an analysis of core competencies and areas suitable to outsource).

2. Needs analysis. This is performed on a particular project to identify the project-specific needs.

3. Supplier assessment. This involves assessing suppliers in order to determine which one best fits the specific needs. This includes not just program management assessments, but a detailed technical review of the engineering competence, depth, infrastructure, and quality. “Choosing the right vendor [supplier] is much like choosing a good partner; the chances are that if you make the right decision from the onset, you will have a potentially lasting relationship, while choosing the wrong vendor [supplier] could damage and thwart a well-intentioned outsourcing project” [1].*

4. Contract and negotiation management. This is the process of negotiating the outsourcing agreement and getting the contract in place. Quality and transition criteria need to be established for each deliverable and milestone. I have seen projects pay for deliverables only to discover they were inadequate and needed to be completely reworked.

5. Project initiation and transition phase. This is the phase where work is transitioned to the supplier. This phase sets the tone for the rest of the project; therefore, it’s important to address issues quickly. This is a particularly critical phase for the technical work. Projects should plan for the appropriate level of technical oversight to ensure communication, issues reconciliation, and acceptability of deliverables.

6. Relationship management. As the project moves into a routine, it is important to keep the customer–supplier relationship healthy. This phase includes evaluating the relationship, resolving problems, managing communications, sharing knowledge, and managing the process.

7. Continuance, modification, or exit strategy. At the end of each project, it’s necessary to evaluate the outsourcing relationship to determine if the arrangement should be continued, modified, or exited. If the decision is to continue, lessons learned should be evaluated and improvements implemented. If the decision is to modify the process, a clear strategy for modification is needed. If the decision is to exit, that too requires a well-planned strategy.

Recommendation 4: Be well informed. Make sure knowledgeable people are available to implement the outsourcing arrangement. If new to outsourcing, it is advisable to get some help. Perhaps an outsourcing strategist or another division of the enterprise can help. There are a multitude of details to consider (e.g., legally binding contracts, import and export laws, intellectual property protection, cultural sensitivity training [for offshore projects], visa applications, equipment transfer, network infrastructure). It is extremely beneficial to get some expert help the first time through and to have a dedicated team to learn all they can from those experts in order to establish a sound foundation.

Recommendation 5: Know your own organization. Prior to launching into an outsourcing arrangement, it is important to know your own organization’s characteristics. Identify strengths, weaknesses, specific needs, organizational maturity, technical expertise, process maturity, etc. From a techni cal perspective, it is important to examine core and noncore competencies. “Core competencies are the combinations of special skills, proprietary technologies, knowledge, information and unique operating processes and procedures that are integrated into the organization’s products and services and are unique differentiators for the organization’s customers” [1]. Noncore competencies are competencies that do not differentiate the organization from competitors and that do not directly impact the organization’s products and services. Noncore competencies are required for daily operations and indirectly impact the organization’s services and products [1]. Most companies choose to outsource noncore competencies first.

Recommendation 6: Determine what to outsource and what to retain in-house. Evaluating core competencies helps with this task; however, once competencies and needs are determined, a strategy to address them is needed. Some common approaches to architecting the project for outsourcing are the following:

• Outsource the entire product systems, hardware, and software.

• Outsource the entire software life cycle and retain hardware and systems development and testing.

• Modularize functionality, so that some functions are done in-house and others are outsourced.

• Separate software development and verification teams (typically development is done in-house and verification is outsourced).

• Outsource software development and verification but use in-house experts to oversee and monitor the work.

When deciding what will be outsourced and what will be retained, consider intellectual property, key functionality that must work fast, interfaces between the various components, and areas highly visible to the customer. These items are typically best kept in-house.

Recommendation 7: Determine the best outsourcing approach. As mentioned in Recommendation #6, there are many possible approaches to outsourcing. One might outsource an entire system, or just have a supplier write and execute the tests. One might outsource to a local company, or to one on the other side of the globe. One might have a contractual arrangement with the supplier, or actually own the offshore subsidy. Mezak identifies six common strategies for software development [3]: (1) in-house engineers, (2) onshore contract outsourcing, (3) offshore contract outsourcing, (4) in-house and offshore blend, (5) offshore subsidiary, (6) build, operate, transfer (start with a contracted team with the option to transfer the engineers to a subsidiary in the future). Since the best approach varies, the options should be evaluated in order to determine the best fit for both short-term and long-term needs.

Recommendation 8: Select supplier wisely. This should be an obvious one; however, having seen several unwise selections, it’s worth a reminder. Once the strengths, weaknesses, and needs of the customer’s team are determined, look for a supplier that meets those needs and complements the customer’s team. Apply the same scrutiny and expectations that would be applied when hiring a new full-time employee for a key position: check references and review resumes. Here are some specific areas to consider:

• Experience on safety-critical software projects

• Experience with DO-178C (and/or DO-178B)

• Similarity of experience to the planned tasks

• Language capabilities (written and spoken)

• Safety domain understanding

• Familiarity with the planned tool suite and environment

• Commitment to following established processes

• Stability and depth of staff

• Capability to support assigned tasks in the time allotted

• Ability to add resources if the project doesn’t go according to plan

• Experience with the specific domain (e.g., navigation, communication, flight controls)

• Track record in the industry (check references, and not just those given by the supplier)

• Experience with the planned project size (such as small, medium, or large project)

• Overlap with the customer’s team’s work day

• Education (get resumes and have a means to ensure the team will be retained on the project)

• Location (safe, secure, easy to reach, etc.)

• Ability to support on-site, as needed (this may involve investigating the visa application process, travel options, extended stay lodging, etc.)

• Respect for intellectual property and proprietary data

• Probability of security vulnerabilities (some countries are perceived as having state-sponsored malware activities)

• Equipment (quality telecommunications system, consistent power, high-speed Internet, office space and equipment quality, video conferencing capabilities, lab space, etc.)

• Cultural compatibility between supplier and customer teams

• Established quality assurance

• Risks (such as government stability, security, conflicts of interest, power outages)

The Outsourcing Handbook identifies the following six common errors when selecting a supplier [1]:

1. Sacrificing needs analysis for a glamorous vendor

2. Evaluating a vendor with cost savings as the decisive factor

3. Poor risk assessment of the vendor

4. Rushing through the process of vendor selection

5. Lack of care in managing interactions between vendors

6. Failing to maintain a balance between using current and new vendors

Keep these in mind when evaluating and selecting an outsourcing partner. It cannot be emphasized enough: Don’t choose based on price alone! It nearly always leads to regrets.

Recommendation 9: Insist on the same team for the duration. Some suppliers start with a solid, mature team, but those engineers are soon moved to another project after the project initiation. Be sure to get the team composition agreement in writing and to verify it throughout the project. Some staff turnover is to be expected and there may be some need to shift engineers to get the right fit. However, it is important to continually monitor the team composition.

Recommendation 10: Prepare the customer team. The customer’s team will need some special attention. Some engineers may be opposed to outsourcing and especially offshoring. Make sure that they understand the reasons for the outsourcing and continually reinforce the fact that they are key to the project and company’s success. Depending on the location of the supplier, it may be necessary to shift schedules to have overlap with the supplier’s team. Also, some cultural sensitivity and diversity training may be needed for both the supplier and customer teams. Throughout the project, listen for feedback and take the input seriously. Most of the time, engineers have valid points that are worth considering. With time, it will become clear whose input can be relied upon and whose requires some filtering; however, everyone should be valued and given the opportunity to share freely.

Recommendation 11: Cross-pollinate teams. Depending on the size and nature of the project, it is often beneficial to have some of the customer personnel on-site at the supplier’s facility and to have some of the supplier’s person nel at the customer’s site. Short-term stays seem to work best (3 months or less). The cross-pollination is valuable for both teams. It helps them to better understand and proactively address the project’s issues.

Recommendation 12: Start small. It is usually best to start the outsourcing adventure with small, well-defined tasks. These tasks can help build the relationship, work out process issues, identify the leaders, and build expertise and confidence. Many companies start with a pilot project—typically a small, noncritical function (or a tool). This allows one to test the communication abilities, test the technical team’s skills, and implement a small but needed function.

Regardless of whether the pilot project is performed or not, it’s still wise to start small. Some companies outsource lower level software first (maybe level C or D). If that goes well, they may outsource more critical and larger projects.

Recommendation 13: Clearly specify expectations. The tasks to be performed and the data to be generated should be documented in the statement of work (SOW). The SOW should be complete, unambiguous, consistent, and measurable. The DO-178C artifacts should be included in the SOW. It’s good to have a technical team and someone knowledgeable of DO-178C and certification review the SOW.

Recommendation 14: For staff augmentation, use common tools and methods. If the outsourced team will be augmenting the customer’s staff, common tools and methods are recommended. The following tools should be common: requirements management and capture tool(s), design tool(s), source code tool(s), source code control tool(s), and problem reporting tool(s). It’s also important to include detailed procedures, examples, and training. Sometimes procedures are clear to the customer’s team who has used them for several years, but they may not be clear to an external supplier’s team. Procedures to be used by the supplier may need to be enhanced.

When outsourcing an entire component or system, it may not be necessary to use the same tool set and processes. However, it will be important to (1) understand the supplier’s processes, (2) ensure that the supplier either delivers or maintains the data and environment long term per the customer and certification authority requirements, and (3) plan and implement a transition strategy for transferring the product and data.

Recommendation 15: Develop a supplier management plan. FAA Order 8110.49 (Change 1) requires a documented supplier management plan [2].

The plan might be part of the Plan for Software Aspects of Certification (PSAC) or a system certification plan, or it may be an enterprise-wide plan. The supplier management plan needs to ensure that suppliers (including staff augmentation and offshored subsidiary partners) comply with all regulations, policy, guidance, agreements, and standards that apply to the project. The plan also explains how all suppliers will be managed. Specific areas of interest to certification authorities are compliance, integration of various artifacts (such as requirements, design, code), certification liaison, problem reporting and resolution, integration, verification, configuration management, compliance substantiation, and data retention [2]. The certification authorities want to know what is being done, by whom, and where. They also want to know how the outsourcing is being managed by the applicant (the entity applying for certification or authorization). Even if the offshore facility is owned by the customer or applicant or is a domestic team used as staff augmentation (as opposed to a supplier), the FAA wants to know how the effort is being managed.*

Recommendation 16: Coordinate the problem reporting process. Software developers and verifiers may not realize the impact of a low-level problem on the system or aircraft safety. Therefore, a common problem report categorization is recommended for all suppliers. Additionally, visibility into the supplier’s problem reports is needed throughout the development and/or verification effort (it is too risky to wait until the end to look at problem reports). Any problem reports that are deferred for the initial certification will require justification and review by all stakeholders, including the certification authorities.

Recommendation 17: Train in key areas. At the beginning of the program it is advisable to identify training needs for the entire team (both the customer and the supplier teams). Some common areas of deficiency are DO-178B/C compliance, robustness testing, low-level requirements capture, documentation of repeatable analyses and procedures, integration, and data and control coupling analyses. For large teams, it may require creativity to get the right training to the right team members. A common strategy is to train the technical leaders first and then have them flow the knowledge down. Computerbased training with a variety of examples can also be valuable. Additionally, an online best practices guide that is regularly updated can be a great way to get the knowledge to the masses.†

Recommendation 18: Pay continual attention to communication. Poor communication is one of the main contributors to project failure; therefore, extra attention should be paid to the communication arrangement and its effectiveness. As Mezak writes: “Using outsourcing is like a marriage.

It takes commitment from both sides to make the relationship work. Good communication is required” [3]. Here are some ways to enhance communication:

• Hold face-to-face meetings for team leads—especially early on and at key phases throughout the project.

• Arrange frequent teleconferences and/or virtual meetings between teams. If it is an offshore supplier, it may be a daily teleconference when the shifts overlap. Video conferencing and network-based document sharing can be helpful.

• Encourage the teams to get to know each other personally, not just professionally.

• Use graphics to communicate difficult concepts. A diagram can often help resolve communication issues.

• Implement some level of on-site presence (i.e., cross-pollinate teams).

• Encourage teams to immediately communicate issues and concerns rather than letting them get out of control.

• Develop an escalation method for when either company believes a significant issue (technical or programmatic) is not adequately addressed.

• Continually look for ways to improve communication, based on lessons learned.

Recommendation 19: Assess and mitigate risks. Software development has a number of risks. Outsourcing may increase some of those risks. As noted earlier, the supplier’s qualifications should be carefully evaluated prior to selection. Any deficiencies should be identified as a risk. As the project evolves, the risks should be consistently reevaluated and risk mitigation established. It does little good to identify a risk and then do nothing about it. Unaddressed risks are like holes in the bottom of a boat. Without attention, they can sink the boat (or project). An ongoing risk assessment with concrete mitigation steps helps prevent sinking.

Recommendation 20: Consider undue burden. Closely related to the risk assessment is the undue burden mitigation. As noted earlier, an offshore supplier in a country without bilateral agreements could become an undue burden to the U.S. government. Therefore, the FAA needs to know that the supplier is being properly overseen and that risks are proactively addressed. Following are some of the common strategies to avoid undue burden:

• Limit offshore work to verification and perform testing for certification credit in the U.S.

• Involve the U.S. team (customer team) in the peer reviews of the data.

• Utilize FAA-authorized designees to perform on-site audits of the supplier’s work to ensure it is compliant.

• Limit offshore work to lower software levels until the certification authorities have confidence in both the customer and supplier teams.

Recommendation 21: Manage it well. Many of the outsourcing fiascos that I’ve observed have been because of poor management. Outsourced projects should be managed by qualified and experienced personnel. Managers in this role ought to have the following skills and abilities: good decision-making ability in uncertain environments, able to embrace change, good relationship-building and communication skills, keen knowledge-management skills, and persistence [1]. Additionally, the technical manager should possess strong technical skills and be able to understand the technical concerns and risks.

A complete list of milestones, with detailed tracking and acceptance of these milestones, is an important tool for managing any software project, including an outsourced project. Dependencies between the teams and tasks should be identified, and the accuracy of status should be ensured. The schedule and milestones should be updated as needed, based on the actual project progress.

Recommendation 22: Be realistic. The technical expectations and schedule expectations should be realistic. It’s not realistic to expect a team of new engineers to produce quality work quickly. In general, it’s advisable to hope for the best but plan for the worst. Success-based planning doesn’t do anyone any good. It might make everyone happy at some point in time, but if it isn’t realistic, that happiness will not last. Good managers deal with reality! Johanna Rothman writes: “Plan for each project to take longer and cost more, especially at the beginning of an outsourcing relationship. My rule of thumb is to increase the estimated time by 30% for the first project. Then monitor the project to see if you need increase that estimate” [4].

Recommendation 23: Keep long-term goals in mind. Some companies get frustrated because their outsourced effort is not turning out as envisioned. It’s important to keep the long-term goals in mind. If neither short-term nor long-term goals seem realistic, they may need to be revamped.

Recommendation 24: Plan for oversight. The certification authorities require insight into the oversight approach, including the management oversight, technical oversight, quality assurance oversight, security controls, and certification liaison oversight. Each of these areas should be detailed in the PSAC and/or some other plan accessible to the certification authorities.

Recommendation 25: Plan for postproject support. The outsourcing arrangement should consider who will perform the post project support. That is, who will maintain the data after the initial certification? This will be a factor when determining how roles and responsibilities are divided and what data go where. For example, if the supplier who developed test cases and procedures will not be utilized after the certification, it will be important to have all tools, data, equipment, etc. delivered so future updates to the test data can be made.

Whoever does the support should be sure to obtain all life cycle data, including open problem reports, Software Configuration Index, and Software Accomplishment Summary prior to making changes.

Recommendation 26: Implement continuous improvement. Every completed outsourcing effort provides numerous lessons learned. It’s important to capture those lessons, to learn from them, and to modify processes accordingly.

26.5 Summary

Outsourcing is here to stay; therefore, it’s important to learn how to implement and manage it effectively. This chapter has identified common issues and some recommendations to help prevent or resolve those issues. Please consult other resources and experts in the field. This is just one person’s perspective. Good luck and enjoy the experience!