10.1.1 What Is Software Configuration Management?

Software configuration management (SCM) is an integral process that goes from the cradle to the grave of the software life cycle. It spans all areas of the software life cycle and impacts all data and processes. Babich writes: “Configuration management is the art of identifying, organizing, and controlling modifications to the software being built by a programming team. The goal is to maximize productivity by minimizing mistakes” [1].

SCM is not just for source code as is commonly believed; it is needed for all software life cycle data. All data and documentation used to produce the software, verify the software, and show compliance of the software requires some level of configuration management. In other words, all of the software life cycle data listed in DO-178C section 11 requires SCM. The rigor of the SCM process applied depends on the software level and the nature of the artifact. DO-178C uses the concept of CC1/CC2 (control category #1 or control category #2) to identify how much configuration control applies to a data item. A data item classified as CC1 must apply all of the DO-178C SCM activities. A CC2 data item, however, may just apply a subset. CC1/CC2 is discussed later in this chapter.

This chapter examines the SCM activities that are required by DO-178C and that are considered best practices in any safety-critical domain. Special attention is given to the problem reporting, change impact analysis (CIA), and environment control processes, since they are important SCM processes that companies often struggle with in the real world.

10.1.2 Why Is Software Configuration Management Needed?

Software development in all domains, including the safety-critical domain, is a high pressure activity. Software engineers are required to develop complex systems with tight schedules and budgets. They are expected to make updates quickly and maintain high-quality software that complies with standards and regulations.

To survive in this brutally competitive world, organizations need some sort of mechanism to keep things under control or total chaos and confusion will result, which could lead to product or project failures and put the company out of business. A properly implemented software configuration management (SCM) system is such a mechanism that can help software development teams create top-quality software without chaos and confusion. [2]

Good SCM helps to prevent problems such as: missing source code modules, inability to find the latest version of a file, reappearance of corrected mistakes, missing requirements, inability to determine what changed and when, two programmers overwriting each other’s work when updating a shared file, and many more. SCM reduces these issues by coordinating the work and effort of multiple people working on the same project. When properly implemented, SCM “prevents technical anarchy, avoids the embarrassment of customer dissatisfaction, and maintains the consistency between the product and the information about the product” [3].

SCM also enables effective change management. Software-intensive systems will change—it’s part of the nature of software. Pressman writes: “If you don’t control change, it controls you. And that’s never good. It’s very easy for a stream of uncontrolled changes to turn a well-run software project into chaos” [4]. Because change happens so frequently, it must be effectively managed. Good SCM provides a means to manage change by: (1) identifying data items likely to change, (2) defining the relationship between the data items, (3) identifying the approach for controlling revisions of data, (4) controlling changes implemented, and (5) reporting any changes made.

Effective SCM provides many benefits to the software team, the overall organization, and the customer, including the following:

• Maintains integrity of the software by using organized tasks and activities.

• Builds confidence with certification authorities and customers.

• Supports higher quality, and hence safer, software.

• Enables the management of life cycle data required by DO-178C.

• Ensures the configuration of software is known and is correct.

• Supports the schedule and budget needs of the project.

• Provides the capability to baseline software and data items.

• Provides the ability to track changes to baselines.

• Avoids confusion and enhances communication among the development team members by providing a systematic approach to data management.

• Helps avoid, or at least reduce, surprises and wasted time.

• Provides a means to identify, record, and resolve problems in the code and the supporting life cycle data.

• Promotes a controlled environment for developing, verifying, testing, and reproducing software.

• Ensures software can be regenerated, even years after the initial development.

• Provides status accounting throughout the project.

• Provides a foundation for developers and customers to make decisions.

• Provides ability to reproduce issues during problem investigations.

• Provides a foundation for process improvement.

• Provides data to confirm when software is complete.

• Supports long-term maintenance.

The risks of not having good SCM are significant. Poor SCM leads to lost time, money, quality, and confidence. I recently consulted on a project with a company that had poor SCM. They had bright engineers, but they could not consistently reproduce the software or the avionics unit. This led to huge delays, increased oversight by the Federal Aviation Administration (FAA), and costly penalties due to the lateness of the product.

Although SCM does not make for exciting reading, it is extremely important. Projects get far too exciting when SCM is not applied.

10.1.3 Who Is Responsible for Implementing Software Configuration Management?

SCM is the responsibility of everyone involved in the software development process. Software development in all domains, including the safety-critical domain, is a high pressure activity. All developers need to be educated on the benefits of good SCM, the risks of poor SCM, SCM best practices, and the specific company SCM procedures. Such training helps engineers do an overall better job. When properly implemented and automated, SCM should not be difficult for the developers to perform on a daily basis.

In the past, SCM was a manual, time-consuming process; however, now, with the availability of good SCM tools, the SCM process can be implemented without a heavy burden on the developers.

But thinking that SCM tools will take care of everything can be a recipe for disaster. Many SCM activities, including change management, build and release management, and configuration audits, require human intervention and judgment. Although SCM tools make these jobs easier, there is no substitute for human intelligence and decision making. [2]

SCM is everyone’s responsibility. It requires communication and teamwork between those who want the software and data, those who produce the software and data, and those who use the software and data. Good SCM starts and ends with communication. Here are some suggestions to encourage communication that will enable effective SCM:

• Ensure the goals and objectives are clearly stated.

• Make sure that all stakeholders understand the goals and objectives.

• Ensure that all stakeholders are cooperating and address any issues that hinder cooperation.

• Ensure that all processes are documented and clearly understood by all stakeholders.

• Provide feedback frequently.

• Make sure data needed for decisions are available.

• Address issues as they arise.

10.1.4 What Does Software Configuration Management Involve?

In addition to the development of the SCM Plan, which is an output of DO-178C Table A-1 objectives, DO-178C identifies six objectives for the SCM process in Table A-8. Each objective is required for all software levels. The six objectives are the following [5]:

• DO-178C, Table A-8, objective 1: “Configuration items are identified.”

• DO-178C, Table A-8, objective 2: “Baselines and traceability are established.”

• DO-178C, Table A-8, objective 3: “Problem reporting, change control, change review, and configuration status accounting are established.”

• DO-178C, Table A-8, objective 4: “Archive, retrieval, and release are established.”

• DO-178C, Table A-8, objective 5: “Software load control is established.”

• DO-178C, Table A-8, objective 6: “Software life cycle environment control is established.”

These objectives are applied to ensure integrity, accountability, reproducibility, visibility, coordination, and control of the software life cycle data as it evolves. In order to satisfy the DO-178C objectives, several activities are required, including: configuration identification, baselining, traceability, problem reporting, change control, change review, status accounting, release, archival and retrieval, load control, and environment control. Each activity is discussed in the next section.

10.2.1 Configuration Identification

“Configuration identification is one of the cornerstones of configura tion management, as it’s impossible to control something whose identify you don’t know” [6]. It is the first activity of configuration management. Configuration identification “identifies items to be controlled, establishes identification schemes for the items and their versions, and establishes the tools and techniques to be used in acquiring and managing controlled items” [2]. Configuration identification provides the starting point for other SCM activities; it is the first major SCM function that needs to be started in a project. It is essentially a prerequisite for other SCM activities, since all the other activities use the output of the configuration identification activity [2].

Configuration identification provides the ability to (1) identify the components of a system throughout its life cycle and (2) trace between the software and its life cycle data. Each configuration item must be uniquely identified. The identification method typically includes a naming convention along with version numbers and/or letters. The identification approach facilitates the “storage, retrieval, tracking, reproduction, and distribution of configuration items” [2].

10.2.2 Baselines

In software, a baseline is the software and its supporting life cycle data at a point in time. The baseline serves as a basis for further development. Once a baseline is established, changes should only be made through a change control process [2,7].

Baselines should be established early in the project; however, bringing all items under configuration control too early can impose unnecessary procedures and slow the developers’ work [2]. “Before a software configuration becomes a baseline, changes may be made quickly and informally. However, once a baseline is established, change can be made, but a specific, formal procedure must be applied to evaluate and verify each change” [4]. So, the question arises, “When should a baseline be established?” There is no hard and fast rule. Usually baselines are established after each life cycle phase at the completion of the formal review that ends the phase. Thus, there is a requirements baseline, a design baseline, a code baseline, etc. The SCM Plan should identify the plan for baselining. It is also important to align the code baseline with the requirements and design that it implements.

Baselines must be immutable. This means the data item(s) is locked down so it cannot be altered. This immutability characteristic is important because a permanent record of the code version (and its supporting data) used to build the release for software testing, flight testing, production, etc. is necessary [8].

“The number and type of baselines depend on which life cycle model the project is implementing. Life cycle models, such as spiral, incremental development, and rapid prototyping, require more flexibility in the establishment of baselines” [9]. Baselines need to be established for configuration items that are used for certification credit [5].

10.2.3 Traceability

Traceability is closely related to baselining and problem reporting. Once a configuration item is baselined, changes are documented (typically through a problem report [PR] and/or change request). Therefore, when a new baseline is established, it must be traceable to the baseline from which it came.

10.2.4 Problem Reporting

Problem reporting is one of the most important SCM activities. PRs are used to identify issues with baselined data, process noncompliances, and anomalous behavior with the software. PRs may be generated to address problems as well as to add or enhance software functionality. Effective problem reporting is essential to managing change and fixing known issues in a timely manner. As noted previously, problem reporting typically starts once a data item has been baselined.

See Section 9.8 for a summary of what a PR contains and recommendations for writing and addressing PRs. Problem reporting is an ongoing process throughout the software development and verification. Problems should be investigated and addressed promptly after identification. Additionally, every effort should be made to write PRs that are understandable by a wide audience, including developers, managers, quality, safety engineers, systems personnel, certification liaison representatives, and certification authorities.

Following are a few topics related to PR management that are especially important to certification.

10.2.5 Change Control and Review

Change control and review are closely related to the problem reporting process because PRs are often used as the vehicle to manage change. Change may occur at any phase of the software development process and is actually a fundamental characteristic of software. Change may occur to add or modify functionality, fix a bug, improve performance, update hardware and the supporting interfaces, improve processes, change the environment (e.g., compiler enhancement), etc. It is relatively easy to change software, which is one of the main advantages of using software; however, it is not so easy to manage it. Without effective change management, chaos occurs. Change management is complex but essential. An effective change management process involves several tasks, including the following:

1. Protection from unauthorized changes. Once a baseline is established, it should be protected from inadvertent and unauthorized changes. All changes to a baseline should be planned, documented, and approved (typically through a change control board [CCB]).

2. Change initiation. All proposed changes to a baseline should be documented. A PR or a change request is the normal vehicle for documenting the proposed change.

3. Change classification. As noted earlier in Section 10.2.4, changes are classified by their impact on the overall system safety, functionality, and compliance.

4. Change impact assessment. In order to properly plan the resources that are needed to implement a change and the amount of reverification required, the impact of the change is documented in a PR or change request. It should be noted that if the software has already been through formal testing or certification, a more formal CIA is usually needed, as discussed later in this chapter.

5. Change review. DO-178C section 7.1.e states that the objective of change review is to ensure that “problems and changes are assessed, approved, or disapproved, approved changes are implemented, and feedback is provided to affected processes through problem reporting and change control methods defined during the software planning process” [5]. Change review involves the review of all changes to determine the potential impact of the change. Typically, a CCB is established as the gatekeeper for the change process. The CCB reviews all PRs and change requests, approves or disapproves the proposed changes, confirms that approved changes are properly implemented and verified, and confirms that PRs or change requests are properly updated throughout the development and verification effort and closed upon completion. Therefore, the CCB should include members who know the software and system well and who have authority and knowledge to make changes. Oftentimes, the customer (e.g., aircraft manufacturer and/or avionics integration team) will be part of the CCB. Earlier, a PR review board was mentioned. In most projects the PR review board and the CCB are the same board. However, for some large projects, they may be separate.

6. Change decision. The CCB evaluates the proposed change and approves, denies, defers, or sends it back for more information.

7. Change implementation. Once a proposed change is approved by the CCB, the change is made as agreed upon in the PR or change request. If the scope of the change is modified during implementation, the PR or change request needs to be updated and may require additional review by the CCB. As noted earlier, change to a configuration item should result in a change to its identification (typically, the version or revision is updated; but sometimes a part number change is needed). Additionally, as previously noted, software changes should be traceable from their origin. Anne Hass writes: “For any configuration item, it must be possible to identify changes in it relative to its predecessor. Any change should be traceable to the item where the change was implemented” [6].

8. Change verification. Once the change is implemented, it is verified. This is usually carried out by an independent review of all changed or impacted artifacts, as well as a rerun of any necessary tests.

9. Change request or PR closure. Once the change has been implemented, verified, and accepted, it may be closed. The CCB is typically responsible for the PR or change request closure.

10. Updated baseline release. The baseline should not be released until all of the approved PRs or change requests have been verified.

10.2.6 Configuration Status Accounting

Configuration status accounting involves recording and reporting information that is needed to effectively manage the software development and verification effort. Reports are generated to inform managers, developers, and other stakeholders about the project’s status. Configuration status accounting provides consistent, reliable, timely, and up-to-date status information that helps to enhance communication, avoid duplication, and prevent repeat mistakes [2]. It often includes reports that provide the following:

• Status of data items, including configuration identification.

• Status of PRs and change requests (including classification, impacted data items, root cause of problems, configuration identification of updated data items).

• Status of released data and files.

• List of baseline contents and differences from previous baseline.

SCM tools are often used to automate the configuration status accounting reports. For the tools to be successful, the data must be accurately and consistently entered. Additionally, to avoid erroneous usage, the tool functionality must be well understood by those using the tool.

Configuration status reports should be planned early in the project in order to properly capture the data. However, as a project progresses, it is sometimes desirable to modify or expand the metrics. For example, when summarizing PRs, one might decide to add fields to identify personnel information (who identified the problem, what team caused the problem, who fixed the problem, etc.) in order to help with staffing needs.

Status reports should be updated at a defined frequency. Most companies either automate them, so they are always current, or update weekly prior to their project team meetings. An out-of-date or erroneous report is useless and can even lead to bad decisions. I was recently involved in a project that frequently provided an out-of-date and inaccurate summary of the PRs. It was very hard for me, as the FAA designee, to determine what data to look at for safety impact. The erroneous data also made me question the accuracy of other data. Therefore, it is important to make every effort to generate accurate status reports; otherwise, they may do more harm than good.

10.2.7 Release

Once a data item is mature, it is released into the formal configuration management system (typically an enterprise-wide library). DO-178C defines release as: “The act of formally making available and authorizing the use of a retrievable configuration item” [5].

Not all data need to go through the formal release process. Some data may just be stored and protected. Typically, the data needed to rebuild and maintain the software (such as requirements, design, code, configuration files, and configuration index) need to be formally released. Supporting data (such as review records and SQA records) just need to be stored with the project records (typically on a secure server). DO-178C identifies the minimum set of data items to be released using the CC1/CC2 categorization. This will be discussed more when we examine control categories later in this chapter (Section 10.2.9).

The release process typically involves review and sign-off of the document/data by key stakeholders (e.g., author, reviewer, software qual ity engineer, certification liaison representative, and data quality). Prior to signing/approving a data item, it should actually be read. This probably seems obvious to many, but I have come across more than a few signed documents that had not been read.

Typically, data (including executable code) are released prior to providing to customer and prior to using for certification credit.

10.2.8 Archival and Retrieval

The archival and retrieval process involves the storage of data (both released data and other data used to support certification) so that it can be accessed by authorized personnel. The archival and retrieval process should consider the following [5]:

• Accuracy and completeness. There should be verification that the proper data are archived. This is often done when preparing and reviewing the Software Configuration Index (SCI) to make sure data in the SCI match what has been archived.

• Protection from unauthorized change. Data should only be updated by authorized personnel and only with the proper change authorization.

• Quality of the storage media to minimize regeneration errors in the short-term and deterioration errors in the long-term. Not all media is suitable for storing safety-critical software.

• Protection during disasters. This is typically carried out by using some kind of off-site storage.

• Readability of data. Type design data need to be available and readable as long as the equipment is in an aircraft. This may require data refresh on a periodic schedule, depending on how reliable the storage media is.

• Archival of supporting tools. If tools are required to read or generate the data, they too may need to be archived; this may include the development and verification environments. If the tools require licensing agreements, those should also be considered.

• Accuracy of retrieval and duplication. When the data are retrieved or duplicated, it must be free from corruption.

• Ability to handle modification without losing data. When data are modified, the previously released and archived data should not be affected.

10.2.9 Data Control Categories

In order to identify how much configuration control is required by data type, DO-178C uses the concept of control category. The concept of control categories is unique to aviation and is explained in DO-178C. There are two control categories defined: control category 1 (CC1) and control category 2 (CC2). The DO-178C Annex A tables identify the applicability of CC1 or CC2 for each data item.^*

CC1 requires maximum control and applies to key certification data and data required for regeneration or accident investigation. Also, for levels A and B more data items are classified as CC1. CC1 requires application of the following SCM processes: configuration identification, baselines, traceability, problem reporting, change control (integrity, identification, and tracking), change review, configuration status accounting, retrieval, protection against unauthorized changes, media selection, refreshing, duplication, release, and data retention [5]. CC2 is a subset of CC1 and requires limited configuration management. It applies to data items that aren’t as critical to executable object code regeneration, such as SCM records and verification records. CC2 requires application of the following SCM processes: configuration identification, traceability to source, change control (integrity and identification), retrieval, protection against unauthorized changes, and data retention [5].

Table 10.2 summarizes DO-178C’s CC1 and CC2 requirements by data item. There are some data items that are always CC1: Plan for Software Aspects of Certification (PSAC), requirements, development trace data, source code, executable object code, parameter data item files, SCI, and SAS. Likewise, there are some data items that are always CC2: PRs, SQA records, verification results, and SCM records. The control category for other data items vary by software level.

The DO-178C control category assignments are considered a minimum. Many companies opt to go above and beyond and require more items to be CC1 than is required by DO-178C. For example, the Software Verification Report is often treated as CC1, even though DO-178C identifies software verification results as CC2.

10.2.10 Load Control

Executable code does nothing until it is actually loaded in the target hardware. Some software is loaded in the factory and some is loaded in the field. (See Chapter 18 for information on field-loadable software.) Following are the key components of a controlled software load process:

• Approved load procedures. Although the actual loading is often outside the software development and DO-178C scope (since loading is often part of the manufacturing process or aircraft maintenance), it needs to be considered during the software approval process. The loading procedures should be developed and verified as part of the DO-178C compliance effort. DO-178C section 11.16.k recommends that the loading procedures be included in the SCI, just as the build instructions are. This was added to DO-178C since DO-178B. In DO-178B, it was not clear where the load procedures were to be documented.

  Table 10.2 Control Categories by Data Type

  

• Load verification. There should be some means to ensure that the software is completely loaded without corruption. This is often carried out by some kind of integrity check, for example, a cyclic redundancy check (CRC).

• Part marking verification. There needs to be some way to identify the loaded software to confirm that the part number and version loaded are consistent with what was approved. Once software is loaded, it should be verified that the identification agrees with the approved data.

• Hardware compatibility. Approved hardware and software compatibility should be documented and adhered to.

10.2.11 Software Life Cycle Environment Control

The software life cycle environment includes the methods, tools, procedures, programming languages, and hardware that will be used to develop, verify, control, and produce the software life cycle data and software product [5]. An uncontrolled environment can lead to numerous problems, including errors in the code, lost data items, elusive errors during testing, inadequate testing, nonrepeatable build process, etc. “Software life cycle environment control ensures that the tools used to produce the software are identified, controlled, and retrievable” [5].

In the planning phase, the software development environment is described in the Software Development Plan, the verification environment is described in the Software Verification Plan, the SCM environment is described in the SCM Plan, and the SQA environment is described in the SQA Plan. The plans describe the procedures, tools, methods, standards, and hardware used to implement processes. It is also recommended to summarize all of the tools in the PSAC.

However, during the planning phase, the details of the environment are often not known (e.g., the compiler may be known but the specific version, options/settings, and supporting libraries may not be known during planning). In order to have a deterministic environment, the details (including specific executable files) must be documented. The details of the software life cycle environment are identified in the Software Life Cycle Environment Configuration Index (SLECI) or an equivalent document. The typical contents of the SLECI are described later (see Section 10.4.3). The SLECI is used to control the environment. There should be processes in place to ensure that the environment identified in the SLECI is what is actually being used by the engineers. This is a commonly overlooked task. The SLECI is frequently not completed until the end of the program. However, in order for the environment to be controlled, it must be completed early in the process. Because the development, SCM, and SQA environments may be known earlier than the verification environment, there may be multiple iterations of the SLECI.

Additionally, the tools used to develop, verify, control, build, and load the software and its data need to be kept under configuration control. Therefore, the tools must be controlled as CC1 or CC2 data. DO-178C requires the executable object code for nonqualified tools to be controlled at CC2, as a minimum. The appropriate control category for qualified tools and their supporting qualification data are specified in DO-330, Software Tool Qualification Considerations. Tool qualification is discussed in Chapter 13; however, be forewarned about the environment used for qualified tools. If your project is using a qualified tool, it is important to verify that the environment in which you are using the tool (the operational environment) is representative of the environment in which the tool was qualified. Otherwise, the tool qualification credit may not be applicable.

10.4.1 SCM Plan

The contents of the SCM Plan were explained in Chapter 5. The SCM Plan should be developed early in the project, so that SCM is properly applied. The SCM Plan should address the items discussed in this chapter, as well as what was discussed in Chapter 5. In addition to the SCM Plan, there may need to be more detailed procedures for some SCM activities. For example, the problem reporting process and the CIA process often require specific details above and beyond what is in the SCM Plan.

10.4.2 Problem Reports

PRs were discussed earlier in this chapter and in Chapter 9. PRs should be identified and kept current throughout the project. It is also important to have a good status summary of PRs to support project management decisions, as well as certification and safety evaluations.

10.4.3 Software Life Cycle Environment Configuration Index

As noted previously and in DO-178C section 11.15, the SLECI is used to document and control the environment for development, build, load, verification, and control. It helps support software regeneration, reverification, or modification. It contains a list of the following [5]:

• Hardware and operating system of the development environment.

• Tools used to develop, build, and load the software (such as compilers, linkers, loaders, requirements management tools).

• Tools used to verify the software (including hardware for testing and operating systems).

• Tools used by SCM and SQA processes.

• Qualified tools and the data used to support the qualification.

Many projects package their SLECI with the SCI. When doing this, it should be noted that multiple iterations of the SCI will be needed, since the environment needs to be controlled before code is built and released.

10.4.4 Software Configuration Index

The SCI provides a listing of all the life cycle data for the software prod uct with its specific configuration (including the source code and executable object code files). If the software contains multiple components, there may be a hierarchy of SCIs; for example, an SCI for each component and then a toplevel SCI to pull all of the components together.

The SCI is an important data item because it essentially “identifies the configuration of the software product” [5] and is important for certification and maintenance. The SCI defines what the software is and what data were used to develop it. It is also one of the three data items that are required to be submitted to the certification authority (in addition to PSAC and SAS).

Per DO-178C section 11.16, the SCI identifies the software product (the part number(s) of the software loaded into the equipment), the executable object code, the source code files (with version information), archive and release media, the life cycle data, build instructions for generating the executable object code, reference to or inclusion of the SLECI, data integrity checks used (such as CRCs), loading procedures and methods, and procedures and methods for user-modifiable software (if used) [5]. If parameter data item files are used, they are also listed, along with any build instructions for creating them. (Parameter data item files are discussed in Chapter 22.)

Typically, an SCI is generated for each baseline of software. For the development baselines, the SCI often just includes a list of the source code files, build instructions, resolved PRs and change requests, and reference to or inclusion of SLECI data. However, for each baseline, it is also recommended to identify the version of plans, standards, requirements, and design that were used to generate the code. The other data can be added as the project progresses, but these items are important to know what drove the code.

I normally examine several SCIs each year. There are a couple of trends that I often see. These are noted in the following for your awareness:

• Oftentimes, the SCI includes the code listing but doesn’t include the other software life cycle data. This seems to be particularly prevalent in companies that have a military background and are used to developing a Version Description Document (which was required by DOD-STD-2167A and other military standards).

• Many teams choose to combine the SLECI and SCI into a single document. This is fine and is even explicitly mentioned in DO-178C. However, teams often do not complete the document until the software is released. If the SLECI is part of the SCI, this means an early version of the document will be needed to define the development environment. Otherwise, it is difficult to prove that the environment was controlled.

10.4.5 SCM Records

SCM Records include additional data used in the SCM process, such as status accounting reports, release records, change records, software library records, and baseline records. The specific records generated vary from company to company. The SCM Plan should either identify the data to be generated or point to company procedures that provide the details.