12.2.1 Best Practices for Coordinating with Certification Authorities

As noted earlier, the communication approach between the applicant and certification authority varies depending on the project details and the authority’s preferences. A majority of the compliance findings are generally delegated to the designees or the delegated organization, with oversight by the certification authority. Following are some suggestions for effective coordination with the certification authorities.

Suggestion 1: Submit a project-level plan. Many organizations submit a ProjectSpecific Certification Plan (PSCP) (or equivalent) that outlines the overall project plans. This plan includes an overview of the project (including the aircraft and systems), identifies the planned software and aircraft electronic hardware activities (with the software and hardware levels and the selected suppliers/developers), defines the proposed means of compliance to the applicable regulations, identifies points of contact, provides a project schedule, etc. Such a plan establishes the framework for the software effort and initiates the communication with the certification authority.

Suggestion 2: Hold an early familiarization meeting. The sooner the certification authority becomes acquainted with the project, the better. The initial familiarization meeting is generally scheduled about the same time the PSCP is ready to submit. The kickoff meeting provides an overview of the project, system, software, schedule, and certification plans. Although the primary purpose of the meeting is to familiarize the certification authority with the planned project, the meeting also provides valuable information to the applicant. The certification authority usually identifies concerns and anticipated certification issues, discusses expected roles and responsibilities, explains communication expectations, and proposes next steps.

Suggestion 3: Identify potential certification and technical issues. Be forthright about any anticipated certification issues or technical challenges. It is important to inform authorities if any new technology or novel approaches are planned. In general, certification authorities don’t like surprises. Surprises can give the impression that someone is hiding something and can reduce the certification authority’s trust level. Therefore, it is important to be honest about potential challenges. It is also important to identify the plans to mitigate or address the challenges. Early disclosure of potential issues will allow the authorities to officially document any concerns. The FAA generates issue papers to identify unique compliance issues. The applicant then provides a response to the issue papers to explain their intended approach for dealing with the issue. Other certification authorities have a similar vehicle for identifying certification issues (e.g., the European Aviation Safety Agency [EASA] uses certification review items [CRIs] and the Transport Canada Certification Agency uses certification memorandums).

Suggestion 4: Provide timely response to issue papers. While issue papers are not pleasant, the earlier they are issued and resolved, the better. Receiving an issue paper late in a program can be extremely disruptive to the project. Once an issue paper is received, it is important to develop a thorough response as quickly as possible. Most issue papers can be anticipated. The FAA has a standard set of software issue papers that are issued on most new certification projects (examples of common topics for issue papers include object-oriented technology, model-based development, object code coverage).^* These general issue papers are sometimes called generic issue papers and serve as a starting point for project-specific issue papers. Additionally, there may be some other project-specific issue papers based on the challenges of the project (e.g., if a large amount of the work is outsourced or offshored, there may be an issue paper to address oversight and problem reporting). The applicant’s response to each issue paper should clarify any of the issues (sometimes the issue paper might present a slightly incorrect understanding of the details) and explain how the issue will be addressed on the specific project. It is important to follow up with the certification authority until the issue paper is closed (i.e., to obtain the certification authority’s approval of the project position). Sometimes, it takes a few iterations to obtain agreement. An unresolved issue paper can be just as risky as a not-yet-issued issue paper.

Suggestion 5: Prepare, brief, and submit PSAC early. As noted in Chapter 5, it is best to prepare and submit the PSAC early. The longer a project waits to submit a PSAC, the higher the risk if the certification authority rejects the plan. For high-risk projects it is strongly recommended to brief the certification authority on the PSAC before submitting it. This allows some informal feedback to adjust the plan if needed or to go ahead and submit it. (See Chapter 5 for details on the PSAC.)

Suggestion 6: Follow up quickly on any questions or issues on the PSAC. The certification authority will often have some questions on the PSAC. Be sure to answer the questions quickly, thoroughly, and accurately. Certification authorities are typically overworked and underpaid, so when they are working on your data, it’s important to be responsive. Otherwise, it could delay the PSAC acceptance.

Sometimes the certification authority will request some updates to the PSAC. It is important to understand the issues; a meeting or phone call can be helpful, if clarification is needed. Oftentimes, the issues are resolved by a brief discussion and possibly an agreement to make a note in the SAS. Such agreements should be documented in writing; an email or meeting minutes are usually sufficient to document such agreements. If the issues still exist after a discussion, present intended updates to the certification authority and get general agreement before making the updates to the PSAC and resubmitting it.

Suggestion 7: Meet with certification authorities throughout the project. For projects that require a high level of certification authority involvement, a periodic meeting with the certification authority should be established. The frequency may vary throughout the project, depending on how things progress. There may be times when a monthly meeting is needed. At other times quarterly or as-needed meetings may be more appropriate. For high-risk projects, designees should be consulted throughout the project and involved in all coordination with the certification authorities.

If the project is a medium or low level of certification authority involvement, most of the coordination will likely be with the designees or company certification liaison group.

The interactions with the certification authorities should focus on the status of the project, any issues that have come up, and the plan for addressing the issues. Action items and due dates should be established and included in the official meeting minutes. It is important to follow through on agreed upon actions.

Suggestion 8: Deal with issues as they arise. Problems rarely go away when they are ignored. They should be addressed promptly before they get out of control. Be sure to inform the designees or certification liaison personnel of any issues that might affect compliance. They will help to determine the preferred approach for informing the certification authorities. When divulging issues to the certification authorities, always have a proposed plan for addressing the issues.

Suggestion 9: Submit any changes or deviations to the certification authority for approval or acceptance. As noted in Chapter 5, the PSAC should identify the plan for handling changes to the approved processes. The certification authorities will likely want to be informed of any changes to processes throughout the software life cycle.

Suggestion 10: Support audits by certification authorities and/or their designees. Software projects are typically audited by the certification authorities or their designees to ensure compliance to DO-178C and the identified issue papers (or equivalent). Audits require preparation and support. More information on what to expect during an audit and how to prepare for it is provided later in this chapter.

Suggestion 11: Submit software compliance data in a timely manner. The SCI and SAS are the two life cycle data items that are typically submitted to the certification authority to demonstrate compliance with the DO-178C and the regulations. These items are normally submitted a few months before the final issuance of the aircraft or engine type certificate. For TSO authorization projects, the SCI and SAS are submitted with the TSO package. Be sure to coordinate the submittal of the data so that the certification authority has adequate time to review and approve it.

In some projects, the certification authority may also request that the software verification results be submitted as part of the compliance data. The expected submittals are usually discussed with the certification authority during the planning phase and documented in the PSAC.

12.4.1 Overview of SOI Audits

In the mid-1990s, the certification authorities in both the United States and Europe began auditing software projects to assess compliance to DO-178B. Unfortunately, it was discovered that many projects were not complying with the objectives. Also, it was noted that there was a huge variance in how certification authorities assessed a project. Because of this, the international Certification Authorities Software Team (CAST) coordinated and identified a software compliance assessment approach. The CAST paper identified four intervention points throughout the project: (1) planning, (2) development, (3) test, and (4) final. The FAA further documented this approach in Order 8110.49 chapter 2 and in a Job Aid entitled Conducting Software Reviews prior to Certification.^* Order 8110.49 explains what the FAA or authorized designees will do and what data will be examined.^† The Job Aid provides a process for how the certification authorities will perform the reviews (also called Stage of Involvement [SOI] reviews). The Job Aid was intended to be a training tool for FAA engineers and their designees to standardize their review process. However, for many projects the FAA now requires that the questions in the Job Aid be completed by the applicant or their designees prior to submittal of certification data. Table 12.1 provides an overview of the SOI number and type, data examined, and the DO-178C objectives assessed.

The Job Aid and FAA Order 8110.49 refer to SOI reviews; however, the term audit distinguishes it from the verification reviews; therefore, the terms audit, auditor, and auditee^‡ are used in this chapter. More information is provided later about conducting a SOI audit (for auditors) and preparing for a SOI audit (for auditees).

12.4.2 Overview of the Software Job Aid

DO-178C sections 9.2 and 10.3 explain that the certification authority may perform reviews; however, no further explanation is given for what these reviews include. Order 8110.49 and the FAA’s Software Review Job Aid provide additional explanation of what will be assessed (Order 8110.49) and how to assess it (the Job Aid). Table 12.1 provides an over view of the four SOI reviews, including the data and DO-178C objectives that are assessed. The Job Aid provides recommendations for how to conduct a SOI audit and provides activities and questions to be assessed during the audit. The Job Aid provides insight into the certification authorities’ thought process and expectations; this insight can help developers prepare for audits and better plan their projects from the beginning. Some companies use the Job Aid to perform self-audits.

The Job Aid is divided into four parts. Part 1 provides an overview, including an introduction to the four SOI audits and the definition of some key terms. Following are the terms that are particularly important to understand [4]:^*

• Compliance is the satisfaction of a DO-178B objective.

• A finding is the identification of a failure to show compliance to one or more of the RTCA/DO-178B objectives.

• An observation is the identification of a potential software life cycle process improvement. An observation is not an RTCA/DO-178B compliance issue and does not need to be addressed before software approval.

• An action is an assignment to an organization or person with a date for completion to correct a finding, error, or deficiency identified when conducting a software review.

• An issue is a concern not specific to software compliance or process improvement but may be a safety, system, program management, organizational, or other concern that is detected during a software review.

Part 2 explains the typical audit tasks. Regardless of the SOI type (planning, development, verification/test, or final), the auditor must prepare for the audit, conduct the audit, document the audit results, summarize the audit in an exit briefing (or sometimes a written executive summary), and conduct follow-up activities (such as preparing the SOI report and ensuring that the findings, observations, and actions are addressed).

Part 3 comprises the bulk of the Job Aid. It summarizes the activities and questions for each SOI audit. A table is included for each SOI audit, summarizing the SOI activities (identified in bold font) and questions that are used to complete the activity. Each question is mapped to the DO-178B objective(s) that the question assesses.^* Table 12.2 provides an example of one SOI 1 activity and some of the supporting questions. Typically, during a SOI audit, a column is added to the table to provide a response to each of the questions (based on the evaluation of the project’s data) and is included in the SOI audit report.

Part 4 of the Job Aid provides some examples of how to summarize the SOI audit results. The typical SOI audit report contents are discussed later in this chapter.

The Job Aid also includes four supplements to help the auditors and the project teams being audited (the auditee) including the following:

• Supplement 1: “Typical Roles and Responsibilities of the FAA Software Team.” This supplement gives a summary of what to expect from the various FAA offices during a software project. Most of the SOI audits are performed by designees, but the certification authorities may be involved in some of the high-risk projects.

• Supplement 2: “Typical Roles and Responsibilities of the Software Designee.” This supplement provides a summary of what a software designee does. This information can be quite helpful to properly utilize designees.

• Supplement 3: “Example Letters, Agendas, and Report.” This supplement provides some example materials for auditors to use when notifying applicants of audits. A sample report is also included.

• Supplement 4: “Optional Worksheets for Reviewers.” This supplement includes some worksheets that can help auditors with their record keeping tasks. These are considered optional and are provided primarily for training purposes.

12.4.3 Using the Software Job Aid

The following sections provide recommendations for auditors and auditees. General recommendations are provided first, followed by a summary of what to expect during a SOI audit and how to prepare for a SOI audit. Having performed or supported hundreds of audits, the subsequent sections are based on my personal experience and lessons learned. Because this information is presented from my personal experience, it is presented using a more interactive tone than other sections of this book.^* Please note that the term applicant/developer is used throughout the upcoming sections to refer to applicant and/or developer. Sometimes the applicant is the software developer; however, many times the software developer is a supplier to the applicant. When the applicant and developer are separate organizations, both should be present at the audit.

12.4.4 General Recommendations for the Auditor

Following are some recommendations for those of you who find yourselves in the auditing role. You may be a designee, a certification authority, a certification liaison engineer, a software quality engineer, or a project engineer.

Regardless of why you’re required to perform the SOI audit, these recommendations are intended to help you do a better job and avoid common mistakes.

Auditor Recommendation 1: Communicate and coordinate the audit plan in advance. Communicate with the applicant/developer’s point of contact a few weeks prior to the audit to address the following:

• Ensure that you understand the applicant/developer’s status and confirm that they meet or will meet the agreed entry criteria for the SOI audit (typical entry criteria are discussed later).

• Coordinate the audit date.

• Prepare the agenda for the on-site or desktop audit^* (the Job Aid Supplement 3 provides some sample agendas).

• Ensure that expectations are clearly communicated.

Let the applicant/developer know who will be on the audit team, how long the audit will take, what kind of meeting space will be needed (one or more rooms), etc. The better the audit plan is communicated ahead of time, the smoother things tend to go on-site. Detailed planning can help make the onsite time more productive and enjoyable.

Auditor Recommendation 2: Get a helper. It is extremely beneficial to have a teammate when assessing compliance, particularly for more formal audits (e.g., SOI audits performed for certification credit). There are several advantages of the team approach. First, it doesn’t take as long; the team can divide the tasks in order to examine more data faster. Second, it provides a more thorough review. Multiple eyes and brains identify more issues, faster. I especially find a team helpful for SOIs 2 and 3 where there is a large amount of data to examine. Third, teamwork provides a witness. I’ve been involved in a few audits where the project fabricated stories about what happened. With a witness, it’s harder to do that.

Obviously, it’s great to have a technically experienced reviewer as a teammate. However, sometimes that is not possible because of the resource limitations. It is still helpful to have someone to act as support even if they are not experienced enough to lead an audit. In fact, it can be a great way to train future auditors. They can help with note taking, reviewing configuration management (CM) and quality assurance data, examining review records, witnessing build and loads, etc.

Auditor Recommendation 3: Be prepared. It is important to prepare for the audit. This may include reading or rereading the plans (if they changed or if it has been considerable time since the last audit), reviewing responses to previous SOI audits, and closing out issues from the previous SOI audits. Unprepared auditors tend to be ineffective.

Auditor Recommendation 4: Be considerate and kind. I find that professionalism and respectfulness are far more productive than power games and fear tactics. As I’ve heard it said: “Honey gets better results than vinegar.” People respond better when they are treated with respect and don’t feel threatened.

Auditor Recommendation 5: Strive to really understand the system, process, and implementation. During the first part of an audit, the applicant/developer will provide a high-level view of their system, software, processes, and status. It’s important to understand the project framework, processes, and philosophy. Read the plans ahead of time and ask questions during the presentations and demonstrations to make sure you understand what is being developed and how it is being developed. The big picture (the forest) is really important to understand before diving into the details (the trees).

Auditor Recommendation 6: Communicate the intent up front and throughout the SOI audit. Each day, be sure to communicate your intentions with the applicant/developer’s team lead. If plans change, let them know. I often schedule a few minutes at the beginning and end of each day to communicate where we are and where we are going. Most applicants/developers are quite responsive if you just let them know what you’re thinking.

Auditor Recommendation 7: Have the applicant/developer walk you through the data initially. If the company is one you are not familiar with or the project is still relatively new to you, it is good to have the applicant/developer walk you through their data at first. Let them show you their requirements, design, code, verification cases, verification procedures, and verification results. Have them demonstrate how their traceability works. You might request that they do one top-down thread (system requirement to high-level software requirement[s] to low-level requirement[s] to code) and one bottom-up thread (code to low-level requirement[s] to high-level software requirement[s] to system requirement[s]). After this, you may feel comfortable examining the data on your own, or you may just have the applicant/developers serve as your driver (i.e., the person who operates the computer per your instructions). I tend to prefer driving on my own, but some auditors like to have the applicant/developer do the driving, so they are free to take notes and ask questions. Either is fine; just be sure to communicate your preference with the applicant/developer.

Auditor Recommendation 8: Be persistent in getting answers. Sometimes it can take multiple attempts to get to the bottom of an issue. Not all issues are obvious. I find that looking at additional data and interviewing multiple sources often helps to get a clearer perspective.

Auditor Recommendation 9: Document as you go. It’s important to keep notes as you perform the audit. If you wait, the details get hazy and may be incomplete. I find it helpful to keep draft notes as I go along and then clean them up each evening. If I wait more than one or two evenings, it gets harder to remember the details.

Auditor Recommendation 10: Don’t lose credibility. When assessing data you will see a lot of potential issues—some of them may end up being major showstoppers and some may just be minor speed bumps. It normally takes a while to calibrate the significance of the issues. If you start picking at the minor details, you may lose the applicant/developer’s ear when you find some truly significant issues. When I’m assessing a relatively new proj ect, I generally keep my thoughts to myself for at least a day or two until I’m more familiar with the data. Sometimes what initially appears to be a big deal dulls in comparison to what I discover as time goes on.

Auditor Recommendation 11: Beware of personalities. One of the challenges of auditing is the interesting personalities that present themselves. Audits are not really pleasant for anyone. Sometimes the stress of the situation brings out the worst in people. Some people are confrontational. Some totally avoid providing any useful information. Some are nervous and high strung. Others make promises and don’t keep them. The personality factor is another reason it’s helpful to have a teammate. Sometimes when if you can’t crack the code to communicate with a person, maybe your teammate can.

Auditor Recommendation 12: Use the objectives as the measuring stick. Throughout the audit, it’s important to use the objectives (either DO-178B or DO-178C, whichever is the means of compliance, and any of the supplements that are applied) as the evaluation criteria. The Job Aid questions and your experience are helpful, but the objectives are what you are evaluating.

Auditor Recommendation 13: Stay focused and follow through. Most audits have at least a few distractions: the data may be unclear, the people may be peculiar, or things may not go according to plan. In these situations, it’s important to stay focused and to get to the bottom of the issue(s). You may need to request additional time to look at the data alone or have someone else walk you through the data.

Auditor Recommendation 14: Communicate potential issues throughout. Once you are familiar with the project and the company’s processes and you’ve gained the respect of the team, it is good to mention issues as you identify them. This gives the applicant/developer the chance to develop a strategy for addressing the issue and possibly discuss it with you before the audit ends. I used to wait until the exit briefing and drop all of the issues in the applicant/developer’s lap at once, but I soon learned that that the brief-andrun approach was not effective. There are a variety of ways to inform the team of noted issues. One option is to hold a brief meeting at the end of the day where you share preliminary issues noted. It’s best to emphasize them as preliminary, since the audit is still under way. Another option is to verbally mention issues as you see them, so the applicant/developer can keep their own list of issues. The important thing is to share the information with the right people throughout the audit, so that the exit briefing and SOI report are merely a summary of what you’ve already discussed.

Auditor Recommendation 15: Reschedule or downgrade, if needed. Sometimes, no matter how much planning and preparation you perform, you will begin an audit and realize that the data just are not ready (stated another way, it does not comply). There are several options in this situation, for example,

• You may continue with the audit and write-up the findings and observations for the applicant/developer to address. You will normally have to redo the audit later.

• You might downgrade the audit to a pre-SOI or an informal assessment.

• You may choose to stop the audit and come back later.

There are other options. The decision will depend on several factors, including personal preference.

Auditor Recommendation 16: Provide the report quickly. Once you finish the SOI audit, it’s important to provide the report as soon as possible, so the applicant/developer can take immediate action. I normally provide a preliminary list of findings, observations, and actions as part of the exit briefing and then send the official report within 1 week. In the list of findings, I find it helpful to distinguish between systemic findings and isolated findings. Systemic findings are findings that were noted in several places and, therefore, require a project-wide action. Isolated findings are noted and need to be fixed but are not widespread; they just require that particular instance to be fixed.

I typically document the findings, observations, actions, and comments in a tabular format. The table is usually presented in landscape view in order to have adequate space for the applicant/developer to respond. Each column is described in the following:

• Issue #—This column identifies the number of the table entry. Like requirements, if a number is assigned, it is best not to renumber; that way cross-references can be made elsewhere in the report. (For example, if a finding is deleted, the number stays and a note is made explaining the deletion.)

• FOCA—This column classifies the issue as a finding (F), observation (O), comment (C), or action (A). Findings and actions require the application/developer to take action. An observation needs a response, but does not necessarily require action. A comment is typically a note that may be useful for compliance assessment. Comments do not require an applicant/developer response or action. When project feedback is needed in order to determine a classification, an “F?” or “?” can be used. Once the project response is received, the classification can be updated.

• Objective—This column notes the DO-178C (or DO-178B) objective(s) related to the issue. Most of the time, DO-178C (or DO-178B) Annex A table number is adequate. If the issue is somewhat controversial, the specific DO-178C (or DO-178B) objective number and section reference may be needed.

• Systemic?—A “Yes” in this column identifies systemic issues.

• Data—This column identifies the document or data item that the issue was noted against. If it is a general issue, and not documentspecific, the word “General” can be used. Somewhere in the report, the data version should be identified as well.

• Issue Description—This column identifies the issue. It should be specific. Include the section, requirement number, code line, excerpt, etc., and clearly explain the issue noted.

• Applicant/Developer Response—This column is used by the applicant/ developer to respond to the issue. It is a good practice to date and initial the responses because sometimes there will be multiple updates before the issue is resolved.

• Evaluation—This column summarizes the SOI team lead’s evaluation of the issue. It is recommended to date and initial the evaluation comment. If the applicant/developer’s response requires them to take additional action, the expected action is explained. If the applicant/developer’s response is acceptable, state this and close the issue. If the applicant/developer’s response is acceptable but requires some future action (perhaps in a later SOI), this should be noted. I normally identify responses that require future action with blue font, so I remember to follow up at the next SOI.

• Status—The status column is also updated by the SOI team lead. Typical entries include: Open, In-Work, Response Acceptable, Closed. The goal is to get all entries to the Closed state.

Table 12.3 provides a summary of the sections that I normally include in a SOI report.

12.4.5 General Recommendations for the Auditee (the Applicant/Developer)

If you are the auditee (the applicant or software developer being audited), here are some suggestions.

Auditee Recommendation 1: Assign a single point of contact to interface with the SOI audit team lead. This is usually the software project lead. If the SOI audit will be by the certification authorities, the point of contact might be the designee (e.g., a DER) for the project. The point of contact will handle the coordination with the SOI audit team lead and the project team.

Auditee Recommendation 2: Coordinate with the SOI audit team lead to understand the plan and agenda. The assigned point of contact coordinates with the SOI audit team lead to ensure that the SOI audit logistics, agenda, and expectations are well understood. Most of the coordination can be handled via email; however, a telephone call or teleconference may be preferred.

Auditee Recommendation 3: Perform a self-audit and be honest about any known problems (and how they are being resolved). Before having a formal SOI audit, it is highly recommended to perform a dry run (pre-SOI) ahead of time. If the formal SOI audit will be performed by a designee, the dry run may be done by SQA and a project team member. If the formal SOI audit will be performed by the certification authority, the designee or certification liaison personnel will likely perform the pre-SOI audit.

Auditee Recommendation 4: Be prepared. Have the data and personnel ready when the SOI audit team arrives. It is frustrating to the auditors if they have to wait for the auditee to gather people and data. First impressions are important in audits and are rarely erased. You don’t want lack of preparation to be the first thing the auditors note. If you are not sure what data are expected, get that clarified ahead of time. Not all software team members need to be in the room for the full audit, but they should be available throughout the event. If a key developer or verifier or the lead engineer will be gone during the audit, be sure to let the audit team lead know ahead of time. The auditing team might prefer to postpone the audit. I joke at how coincidental it is that the key personnel have all-day dentist appointments, vacation, or grandma’s funeral (for the third time) when I arrive for an audit. Obviously, scheduling conflicts arise, but do everything you can to have the key people there. Normally, SQA and team leads participate in the entire audit.

Auditee Recommendation 5: Present the project accurately and concisely. In a routine audit, the applicant/developer has a half day to tell the project’s story. Use the time wisely, since it will set the tone for the rest of the audit. Provide an accurate and concise overview of your system, software architecture, processes, and known issues. If it is a SOI 3, explain the testing approach. Provide a brief walkthrough of requirements, design, code, and testing data to help the SOI audit team understand the project’s approach and data layout. Provide the background and overview that will enable the auditors to effectively do their job. Be as concise but complete as possible. Auditors may get annoyed if they suspect someone is wasting their time. One company that I audited tried to stretch their presentation out over the entire 3-day period, even though the agenda was quite clear that they needed to finish at noon on the first day. They acted surprised when I explained that I wanted to see the data after lunch.

Auditee Recommendation 6: Be cordial, cooperative, and positive. Audits are not pleasant for anyone, but they can be handled with professionalism.

Applicant/developer teams that are cordial, cooperative, and positive will almost always end up with a better SOI report. Attitude does count. An uncooperative attitude draws suspicion and scrutiny, but a cooperative attitude and behavior develops trust.

Auditee Recommendation 7: Look at the experience as an opportunity to improve. As in every field, there are a few loose cannons out there who perform SOI audits. However, most of the SOI auditors have vast experience. They have seen numerous projects and can be a wealth of knowledge. It is best to look at the SOI audit as a learning opportunity, rather than a torture chamber. The best companies I’ve worked with are eager to learn how to improve.

Auditee Recommendation 8: Make sure the team thoroughly understands any findings, including the DO-178C (or DO-178B) foundation. It is important to understand the findings and actions documented by the SOI audit team. Occasionally, there will be a finding that is really an opinion. If you suspect this is the case, tactfully inquire about the DO-178C (or DO-178B) objective that is not being satisfied. Also, be sure you understand which findings are systemic (apply across the project) and which ones are isolated, since this significantly impacts the responses. When in doubt, ask for clarification.

Auditee Recommendation 9: Take findings seriously and follow up. By definition, a finding is a noncompliance that must be addressed prior to certification. Therefore, action is required.

Auditee Recommendation 10: Document how the team will address all findings, actions, and observations. As soon as possible, begin working on a response to the SOI issues. Typically, a SOI audit report will be generated with a list of findings, actions, and observations. Oftentimes, the list will be in a table format, so you can just add a column with the project’s response. If the list is not provided in a table format, I recommend putting it in a table format (some companies use a spreadsheet because it provides a convenient way to sort and filter). Be sure to respond to all findings, actions, and observations. Observations don’t have to be addressed but they do need a response. If you have any questions, get clarification from the SOI audit team lead or your designee. Once a response to all the findings, actions, and observations has been prepared, provide the response to the SOI audit team lead to get feedback. Many times, it takes a few iterations to reach agreement on all issues. In some situations it may be more expedient to schedule a teleconference or a meeting to discuss the responses rather than going back and forth on email.

12.4.6 SOI Review Specifics

This section examines more details on each of the four SOI audits. The following topics are discussed:

• Typical entry criteria for a SOI audit (items to be done before a SOI audit can be performed).

• What the SOI audit team typically does during the audit.

• How to prepare for a SOI review.