Chapter 10
An Integrated Framework for Hierarchical Internal Controls

10.1 Internal Control Principles and the Integration of Internal Control, Risk Management, and Governance

This chapter explores how internal controls for TRIO organizations can be fully integrated with enterprise risk and opportunity management (EROM). It provides an extension and follow-up to earlier sections in this book (notably Sections3.6 and 4.7) and is intended to be responsive to the recently issued requirements of the Office of Management and Budget (OMB) in Circular A-123 (OMB 2016) concerning EROM and internal controls. It also recommends innovative approaches that exceed the minimum OMB requirements in several areas.

In a nutshell, the following key principles for internal controls are advocated in this book:

  • Internal controls should be derived from the organization's strategic objectives, tactical objectives, and core standards of operation and from considerations of the risk and opportunity drivers that affect the organization's ability to meet those objectives and standards.
  • The drivers are determined from the factors that most significantly affect aggregate risks and opportunities rather than just from individual risks and opportunities.
  • The identification and evaluation of internal controls focus largely on protection of the assumptions and/or correction of the actual and potential weaknesses that need to be addressed for the aggregate risks and opportunities to be effectively and efficiently controlled within the decision maker's risk tolerance and opportunity appetite.
  • Internal controls should be the result of organizational staff members thinking about what could go wrong and what can be done to monitor and prevent it from going wrong.

Originally, control theory was developed for mechanical systems to ensure that they achieve their operational objectives while operating within predetermined constraints. In the case of mechanical systems, the constraints normally are defined by physical variables such as pressure, temperature, and flow rate. The system is regulated by feedback loops, referred to as control loops, which ensure that these variables are monitored and adjusted when needed through mechanical actuators to keep them within their designed operating range.

The implementation of control theory based on the use of control loops has been applied more recently in the context of internal control for organizations. The following principle of internal control for organizations is posited herein and developed formalistically in Section 10.2.1:

  • Internal control loops can and should generally be derived and implemented in a hierarchical manner, wherein each level of the organization contributes to the overall internal control framework in a synergistic manner.1

Hierarchical control structures are analogous to organizational structures in a large company or agency. They contain a primary control loop, similar to the executive function in a company or agency, and a nested series of subordinate control loops, similar to a company's or agency's hierarchical structure of organizational entities and subunits. However, the control loop hierarchies do not exactly mirror the organizational hierarchies, as they are designed to address control needs that may transcend organizational hierarchies. The following principle applies here:

  • There should be a clear mapping between the internal control structures within an organization and the organizational management structures so that the roles and responsibilities for formulating and implementing controls are clearly defined in terms of organizational entities.

That said, the principles for creating hierarchical control structures are similar to the principles for creating hierarchical organizational structures in the following senses:

  • Hierarchical internal control structures should promote the defining of roles and responsibilities so as to ensure that all important controls have a champion or owner as well as an oversight authority.
  • They should cause lower-level organizational units to be aware that they have control responsibilities that are intended to support the higher-level controls that exist at higher levels of the organization.
  • They should be well-suited for integration with hierarchical EROM structures.

As discussed earlier in Sections 2.4.2 and 4.8.2, organization-wide risk and opportunity management works best when EROM teams are established for each unit of the organization, with frequent communication between the teams both horizontally and vertically. The same is true for internal control structures. In this way, risks and opportunities and internal controls can be treated consistently at each level of the organization and can be easily aggregated from bottom to top.

The relationship between the key organizational management functions of internal control, risk management, and governance is discussed in OMB Circular A-123. As illustrated in Figure 2.8, the OMB Circular views internal control as being a part of program/project risk management, program/project risk management as being a part of enterprise risk management, and enterprise risk management as being a part of governance.

For purposes of this discussion, the principal elements contained within each of these organizational management functions and the interfaces between them are illustrated in Figures 10.1 and 10.2. Figure 10.1, which is notionally similar to Figure 2.6 but reformatted to highlight overall relationships in the present context, applies to strategic planning, whereas Figure 10.2, which is notionally similar to Figure 2.7, applies to performance evaluation. Both are equally important functions of the agency that are subject to governance, risk management, and internal control.

c10f001

Figure 10.1 Conceptualization of the relationship between governance, risk management, and internal controls: strategic planning

c10f002

Figure 10.2 Conceptualization of the relationship between governance, risk management, and internal controls: organizational performance evaluation

The relationships between the various levels of the organization in conducting these organizational management functions are depicted in a very simplified form in Figure 10.3. The reader may want to refer back to Section 3.6.6 for a comparison of the bidirectional integration of internal controls and EROM advocated in this book to the more unidirectional approach advocated by COSO.

c10f003

Figure 10.3 Simplified schematic of the interfaces between organizational management functions and organizational management levels

10.2 Methodological Basis

10.2.1 Hierarchical Control Loops

A control loop is, by Merriam-Webster's definition, “an operation, process, or mechanism that is regulated by feedback.” The feedback loops in Figures 10.1 and 10.2 are used to inform a process but not specifically to regulate it, and are therefore not control loops by Webster's definition.

Figure 10.4 shows a simplified representation of a control loop based on a model presented by Leveson (2011). The controller obtains information about the process state from measurable variables and uses this information to initiate action by manipulating controlled variables to keep the process operating within predefined limits or set points. A separate control loop of this form is typically provided for each of the organization's near-term, mid-term, and long-term objectives that are considered key to its mission.

c10f004

Figure 10.4 Standard control loop form

The original formulation of control loops was intended for mechanical systems, such as that shown in Figure 10.5. In this simple example, a forced-air heating/cooling system (control mechanism) is used to keep the temperature of an enclosed space within comfortable temperature limits (control process output).

c10f005

Figure 10.5 Example simple control loop for a mechanical system

The temperature (measured variable) is measured by a thermometer (monitoring mechanism) and the heating/cooling system is actuated by a thermostat (controller). The system controls the flow rate and temperature of air going into the enclosure (controlled variables). The necessary flow rate and air temperature are affected by the occupant load, equipment heat load, outside temperature, and amount of insulation in the walls (process inputs). The system is subject to various hazards that could lead to its failure, including adverse weather and mechanical/electrical breakdown (actual/potential disturbances).

When used for purposes of internal control in a TRIO enterprise, it is useful to think of the internal control structure as being composed of a hierarchy of control loops, as shown in Figure 10.6. The important features are as follows:

c10f006

Figure 10.6 Example form of a hierarchical system of internal control loops

  • The primary control loop emanates from an objective of the organization, derives from risk or opportunity drivers, and follows the format of Figure 10.4.
  • Each monitoring and control mechanism in the primary control loop is considered to be an activity that may in itself need to be controlled via a secondary control loop. A secondary control loop would be necessary if there are risky elements of the activity (referred to in the figure as “control needs”) that need to be monitored and, when appropriate, acted upon.
  • Likewise, each monitoring and control mechanism in the secondary control loop is considered to be an activity that may need to be controlled via a tertiary control loop, based on risks that apply to it. The development of the hierarchy of control loops may continue to lower levels if needed.

10.2.2 RACI Matrices

Each activity in the hierarchy that spawns a lower-level control loop is assigned to an appropriate entity in the organization, has appropriate management oversight, and is communicated among other entities that have an interest or obligation that interfaces with that activity. A favored means for doing this is to develop a RACI matrix (Smith, 2005), which defines the organizations and people who are Responsible, Accountable, Consulted, and Informed on each activity (hence the acronym). An example form for a RACI matrix is shown in Table 10.1.

Table 10.1 Example form of a RACI matrix

RACI Definitions:
Responsible = person or role responsible for ensuring that the item is completed
Accountable = person or role responsible for actually doing or completing the item
Consulted = person or role whose subject matter expertise is required in order to complete the item
Informed = person or role that needs to be kept informed of the status of item completion
Control Loop Monitoring or Control Activity Responsible Accountable Consulted Informed
Org Person Org Person Org(s) Person(s) Org(s) Person(s)

10.3 Examples

10.3.1 Example 1: Institutional Responsibility for Risk Management and System Safety

In this example, we consider a safety and mission assurance (SMA) organization within a TRIO enterprise whose main responsibilities are to promote, promulgate, and help implement a unified risk management structure and a unified system safety structure within the enterprise. The SMA organization is charged with developing and helping to implement policy and standards that account for all strategically important enterprise activities within the risk management and system safety structures, including all applicable risks and interactions, and integrating the management of risks and system safety across organizational boundaries. Its charter includes the application of engineering and management principles, criteria, and techniques to optimize risk management and safety within the constraints of operational effectiveness, time, and cost.

Based on this charter, a rather simplified primary control loop for the SMA organization's role in risk management and system safety is shown in Figure 10.7. The principal controller in this case is the SMA organization. The control activities (or mechanisms) selected here for display include the development and updating of risk management and system safety policies, procedures, standards, and guides; provision of risk management and system safety training; counseling on risk mitigation options; and reliance on a technical authority to ensure technical quality.

c10f007

Figure 10.7 Example primary control loop for the objective of improving risk management and system safety methodology and practice within the enterprise

For the control loop to work in the best manner, secondary control loops may be needed for several, if not all, of the monitoring activities and control activities of the primary control loop. For example, the control activity “Develop/update RM & SS policies, procedures, standards, guides” in Figure 10.7 requires a control loop to ensure that the activity is initiated when needed and that the products fulfill the needs of the enterprise. In order to develop a meaningful secondary control loop for this activity, it is useful first to devise a process diagram depicting the various activities that are involved in developing and updating RM and SS policies, procedures, standards, and guides. Such a diagram is shown in the lower half of Figure 10.8 on Pages 288 and 289. A control loop that reflects the principal elements in the process diagram is presented in Figure 10.9 on Pages 290 and 291. The control activities for the secondary loop include ensuring that the stakeholders' views are represented, that the policy and procedures are applicable and comprehensive, and that that they are compatible with other policies in the TRIO enterprise.

Similarly, there can be tertiary control loops designed to ensure that each activity in the secondary loop works as intended. For example, the monitoring activity “Oversight of review” in Figure 10.9 may require a control to ensure that the competency of the review group remains high despite possible turnovers within the group, changes in the areas of experience and expertise needed to be a reviewer, and/or changes in the manager that has the authority to grant budget and allocation of human resources toward the review process. A process diagram and tertiary control loop resulting from these considerations is shown in Figure 10.10.

The results of the hierarchical devolvement of control loops may be summarized in a table similar to Table 10.2. Persons and organizations responsible for, accountable for, consulted about, and informed about the status of and results from each monitoring and control activity may be documented in a table similar to Table 10.3. There needs to be an entry in each cell of Table 10.3 in order for the control structure to be considered free of gaps.

10.3.2 Example 2: NASA Commercial Crew Program Risk-Based Assurance Process and Shared Assurance Model

As discussed in Section 7.3.1, the objective of the NASA Commercial Crew Program (CCP) is to develop the capability to use a commercially provided space system to transport crew to low-Earth orbit, including to the International Space Station (ISS). To operate within mandated constraints, the CCP has adopted an approach designed to promote safety and reliability while minimizing schedule slippages and cost overruns. As mentioned in Section 7.3.2, the approach is referred to as a risk-based assurance (RBA) process utilizing a shared assurance model (Canfield 2016; Kirkpatrick 2014). The shared assurance approach takes advantage of each support organization's unique skills and areas of expertise while minimizing organizational overlap and still maintaining an appropriate level of checks and balances. Based on a set of factors that include design and process complexity, degree of maturation, past performance, and expert judgment, a decision is made about which of the hazards identified by the provider(s) need to be analyzed by NASA personnel and which do not. The ones that need to be overseen by NASA are assigned to NASA's list of product assurance actions (PAAs).

c10f008a c10f008b

Figure 10.8 Process diagram for the selected control activity: “Develop and update risk management and system safety policies, procedures, standards, and guides”

c10f009a c10f009b

Figure 10.9 Secondary control loop for the selected control activity: “Develop and update risk management and system safety policies, procedures, standards, and guides”

c10f010a c10f010b

Figure 10.10 Process diagram and tertiary control loop for the selected control activity: “Develop and update RM and SS policies, procedures, standards, and guides”

Because the RBA process and shared assurance model is a new approach to assurance at NASA, the approach itself needs to have a set of implementation controls to ensure that it is effective and efficient. The controls particularly need to address the quality and degree of rigor applied to the communication between those responsible for assuring individual parts of the system and assure that cross-cutting and system-wide concerns are adequately addressed.

Table 10.2 Example summary chart of cascading activities, weaknesses, and controls for the SMA organization example

Objective Risk or Opportunity Driver Primary Control Loop Activity(M/C)* Actual or Potential Weakness Secondary Control Loop Activity(M/C)* Tertiary Control Loop Actual or Potential Weakness Activity(M/C)*
Improve risk management and system safety methodology and practice Out-of-date documentation of RM & SS policies, procedures, standards, guides (risk) Fig. A Develop/ update RM & SS policies, procedures, standards, guides (C) Stakeholders' and users' views may not be accurately represented Fig. B Communicate regularly with stakeholders and users (M)
Ensure stakeholders' views are represented (C)
Policies and procedures may be incomplete, incompatible, or inapplicable Fig. C Oversee review (M) Fig. D Insufficient budget and/or lack of support for matrixed staff Ensure budget and management support (C)
Monitor changes in responsible management (M)
Review team qualifications and/or experience incompatible with need Ensure qualifications of review team members (C)
Ensure experience in all relevant areas (C)
Monitor changes in review team membership or time availability (M)
Ineffective or inefficient implementation of RM & SS policies, etc. (risk) Fig. A Improve RM & SS training (C) Etc.
Etc.

*M = Monitoring Activity, C = Control Activity

Table 10.3 Example RACI chart for the SMA example

Control Loop Monitoring (M) or Control (C) Activity Responsible Accountable Consulted Informed
Org Person Org Person Org(s) Person(s) Org(s) Person(s)
Fig. B Communicate regularly with stakeholders and users (M) X X X X X
X
X
X
X
X
X
X
Ensure stakeholders' views are represented (C) X X X X X
X
X
X
X
X
X
X
Fig. D Ensure budget and management support (C) X X X X X
X
X
X
X
X
X
X
Monitor changes in responsible mgmt. (M) X X X X X
X
X
X
X
X
X
X
Ensure qualifications of review team members (C) X X X X X
X
X
X
X
X
X
X
Ensure experience in all relevant areas (C) X X X X X
X
X
X
X
X
X
X
Monitor changes in review team membership or time availability (M) X X X X X
X
X
X
X
X
X
X
Etc.

Figure 10.11 depicts a candidate primary control loop that could serve this purpose while maintaining the RBA process and shared assurance model intact. The control loop is labeled as “candidate” because it attempts to provide the needed monitoring and control actions but has not yet been vetted. The inputs to the control process are the existing communication protocols, risk set points for assigning individual hazards to BAAs, mitigations and associated internal controls upon those mitigations, risk scores for individual hazards, and aggregate risk for each of the CCP's objectives. The outputs are a revised set of communication protocols, risk set points, mitigations with associated internal controls, residual risk scores for each hazard, and residual aggregate risk for each objective. The primary monitoring activities include analyses (interface hazard analyses and integrated safety cases), monitoring teams (the RBA team and the Safety Technical Review Board), and independent technical authorities (ITAs). The primary control activities include communication protocols (scheduled interactions among entities), adjustments in risk set points, introduction of mitigations, and investiture of an organization-wide risk management team that includes both provider and NASA personnel.

c10f011

Figure 10.11 Example primary control loop for CCP's objective of achieving acceptable safety within schedule and budget using the RBA process and shared assurance model

As demonstrated earlier for Example 1, it is possible to devise secondary, tertiary, and even lower-level control loops for each monitoring and control activity. While attempting to do so in this example would be premature (mainly because it would need a level of detail that would require the participation of CCP personnel), it is possible to provide an outline of the kinds of monitoring and control variables and activities that might be included in these lower-level control loops. Such an outline is presented in Table 10.4.

Table 10.4 Candidates for secondary and tertiary control loops for CCP risk-based assurance process and shared assurance model

Controlled Process Monitored Variables Control Mechanisms
1. Maintenance of competent personnel
  • Staff attrition
  • Staff availability
  • Staff areas of expertise
  • Hiring program
  • Incentives
  • Staff-matrixing
  • Training programs
2. Maintenance of a competent review team
  • Reviewer availability
  • Reviewer expertise
  • Reviewer affiliations
  • Inter-organizational agreements
  • Sharing agreements with external entities
3. Maintenance of needed policy and procedures
  • Changes in environment, organization, or technology
  • Implementation success
  • Periodic reviews of environment, organization, and technology
  • Budget allocation for document updates
4. Maintenance of IT capability
  • Bandwidth needs
  • Cyber threats
  • Aging of IT equipment
  • Periodic reviews of cyber threats
  • Budget allocation for IT system improvement
5. Provisions for incentivizing good communication
  • Implementation of communication protocols
  • Attendance at interdisciplinary meetings
  • Communication training
  • Management support for cross-organizational communication
6. Access to externally produced information
  • Links to information sources
  • Access to proprietary or sensitive information
  • Periodic reviews of available sources
  • Nondisclosure agreements
7. Effective and timely precursor and anomaly tracking
  • Efficiency/effectiveness of tracking system
  • Adherence to best practices
  • Comprehensive, accessible, and user-friendly database
  • Reporting protocol for precursors and anomalies
8. Effective and timely precursor and anomaly tracking
  • Efficiency/effectiveness of tracking system
  • Adherence to best practices
  • Comprehensive, accessible, and user-friendly database
  • Management oversight
9. Effective and timely implementation of waivers/ exceptions
  • Time taken for related waivers and exceptions
  • Lessons learned from other programs
  • Incentives for timely waiver submittals and approvals
  • Protocols for elevation of waiver decisions
10. Effective and timely approval process
  • Time taken to approve related submittals
  • Approval backlog
  • Clear assignment of approval responsibility
  • Close coordination with approval authority

10.4 Incorporation of Internal Control Principles into the Control Loop Approach

Existing frameworks for internal control emphasize not only the strategic and/or top-level objectives of the organization as the starting point for the development of control loops but also principles of good practice. For example, the GAO Green Book (GAO 2014) provides 17 principles that are intended to guide the evaluation of an organization's internal controls. The formulation in the Green Book is similar to that in the COSO Internal Controls Integrated Framework (COSO 2013), which also is based on adhering to 17 principles. The Green Book essentially tailors the COSO principles to make them applicable to government agencies.

As shown in Table 10.5, these principles cover integrity and ethical values, management oversight, delegation of responsibility and authority, staff competency, development of policies, communication, and other practices. Underlying each principle is a set of means for accomplishment (referred to in the Green Book as attributes). As shown in Table 10.6, these means for accomplishment principally consist of a set of best practices and standards.

Table 10.5 GAO green book principles for internal control (GAO 2014)

  1. The oversight body and management should demonstrate a commitment to integrity and ethical values.
  2. The oversight body should oversee the entity's internal control system.
  3. Management should establish an organizational structure, assign responsibility, and delegate authority to achieve the entity's objectives.
  4. Management should demonstrate a commitment to recruit, develop, and retain competent individuals.
  5. Management should evaluate performance and hold individuals accountable for their internal control responsibilities.
  6. Management should define objectives clearly to enable the identification of risks and define risk tolerances.
  7. Management should identify, analyze, and respond to risks related to achieving the defined objectives.
  8. Management should consider the potential for fraud when identifying, analyzing, and responding to risks.
  9. Management should identify, analyze, and respond to significant changes that could impact the internal control system.
  10. Management should design control activities to achieve objectives and respond to risks.
  11. Management should design the entity's information system and related control activities to achieve objectives and respond to risks.
  12. Management should implement control activities through policies.
  13. Management should use quality information to achieve the entity's objectives.
  14. Management should internally communicate the necessary quality information to achieve the entity's objectives.
  15. Management should externally communicate the necessary quality information to achieve the entity's objectives.
  16. Management should establish and operate monitoring activities to monitor the internal control system and evaluate the results.
  17. Management should remediate identified internal control deficiencies on a timely basis.

Table 10.6 GAO green book means of accomplishment for principle 1 (GAO 2014)

  1. The oversight body and management should demonstrate a commitment to integrity and ethical values.
    1. The oversight body and management demonstrate the importance of integrity and ethical values through their directives, attitudes, and behavior.
    2. The oversight body and management lead by an example that demonstrates the organization's values, philosophy, and operating style. The oversight body and management set the tone at the top and throughout the organization by their example, which is fundamental to an effective internal control system. In larger entities, the various layers of management in the organizational structure may also set “tone in the middle.”
    3. The oversight body's and management's directives, attitudes, and behaviors reflect the integrity and ethical values expected throughout the entity. The oversight body and management reinforce the commitment to doing what is right, not just maintaining a minimum level of performance necessary to comply with applicable laws and regulations, so that these priorities are understood by all stakeholders, such as regulators, employees, and the general public.
    4. Tone at the top can be either a driver, as shown in the preceding paragraphs, or a barrier to internal control. Without a strong tone at the top to support an internal control system, the entity's risk identification may be incomplete, risk responses may be inappropriate, control activities may not be appropriately designed or implemented, information and communication may falter, and results of monitoring may not be understood or acted upon to remediate deficiencies.
    5. Management establishes standards of conduct to communicate expectations concerning integrity and ethical values. The entity uses ethical values to balance the needs and concerns of different stakeholders, such as regulators, employees, and the general public. The standards of conduct guide the directives, attitudes, and behaviors of the organization in achieving the entity's objectives.
    6. Management, with oversight from the oversight body, defines the organization's expectations of ethical values in the standards of conduct. Management may consider using policies, operating principles, or guidelines to communicate the standards of conduct to the organization.
    7. Management establishes processes to evaluate performance against the entity's expected standards of conduct and address any deviations in a timely manner.
    8. Management uses established standards of conduct as the basis for evaluating adherence to integrity and ethical values across the organization. Management evaluates the adherence to standards of conduct across all levels of the entity. To gain assurance that the entity's standards of conduct are implemented effectively, management evaluates the directives, attitudes, and behaviors of individuals and teams. Evaluations may consist of ongoing monitoring or separate evaluations. Individual personnel can also report issues through reporting lines, such as regular staff meetings, upward feedback processes, a whistle-blowing program, or an ethics hotline.
    9. Management determines the tolerance level for deviations. Management may determine that the entity will have zero tolerance for deviations from certain expected standards of conduct, while deviations from others may be addressed with warnings to personnel. Management establishes a process for evaluations of individual and team adherence to standards of conduct that escalates and remediates deviations. The oversight body evaluates management's adherence to the standards of conduct as well as the overall adherence by the entity. Management addresses deviations from expected standards of conduct timely and consistently. Depending on the severity of the deviation determined through the evaluation process, management, with oversight from the oversight body, takes appropriate actions and may also need to consider applicable laws and regulations. The standards of conduct to which management holds personnel, however, remain consistent.

Work by Nancy Leveson et al. at MIT (2005) also bases the starting point for an internal control framework on a set of principles (referred to by Leveson et al. as system safety requirements), and for each principle, means of accomplishment (referred to by Leveson et al., as constraints). As shown in Table 10.7, the principles and means for accomplishment are largely based on best practices.

Table 10.7 MIT-conducted NASA independent technical authority study: system safety principles for internal control and means of accomplishment (Leveson et al. 2005)

  1. Safety considerations must be first and foremost in technical decision-making.
    1. State-of-the art safety standards and requirements for NASA missions must be established, implemented, enforced, and maintained that protect the astronauts, the workforce, and the public.
    2. Safety-related technical decision making must be independent from programmatic considerations, including cost and schedule.
    3. Safety-related decision making must be based on correct, complete, and up-to-date information.
    4. Overall (final) decision making must include transparent and explicit consideration of both safety and programmatic concerns.
    5. The Agency must provide for effective assessment and improvement in safety-related decision making.
  2. Safety-related technical decision making must be done by eminently qualified experts, with broad participation of the full workforce.
    1. Technical decision making must be credible (executed using credible personnel, technical requirements, and decision-making tools).
    2. Technical decision making must be clear and unambiguous with respect to authority, responsibility, and accountability.
    3. All safety-related technical decisions, before being implemented by the Program, must have the approval of the technical decision maker assigned responsibility for that class of decisions.
    4. Mechanisms and processes must be created that allow and encourage all employees and contractors to contribute to safety-related decision making.
  3. Safety analyses must be available and used starting in the early acquisition, requirements development, and design processes and continuing through the system life cycle.
    1. High-quality system hazard analyses must be created.
    2. Personnel must have the capability to produce high-quality safety analyses.
    3. Engineers and managers must be trained to use the results of hazard analyses in their decision making.
    4. Adequate resources must be applied to the hazard analysis process.
    5. Hazard analysis results must be communicated in a timely manner to those who need them. A communication structure must be established that includes contractors and allows communication downward, upward, and sideways (e.g., among those building subsystems).
    6. Hazard analyses must be elaborated (refined and extended) and updated as the design evolves and test experience is acquired.
    7. During operations, hazard logs must be maintained and used as experience is acquired. All in-flight anomalies must be evaluated for their potential to contribute to hazards.
  4. The Agency must provide avenues for the full expression of technical conscience (for safety-related technical concerns) and provide a process for full and adequate resolution of technical conflicts as well as conflicts between programmatic and technical concerns.
    1. Communication channels, resolution processes, and adjudication procedures must be created to handle expressions of technical conscience.
    2. Appeals channels must be established to surface complaints and concerns about aspects of the safety-related decision-making and technical conscience structures that are not functioning appropriately.

Principles such as those in the Green Book and the MIT work are usually qualitative in nature, and the success of an organization in satisfying the principles is usually measured using a qualitative rating scheme (e.g., green, yellow, or red). In order to obtain a rating for the status of each principle, the organization first rates the status of each means of accomplishment (e.g., using the green, yellow, and red format). An aggregation scheme is used to synthesize the rating of the principle from the rating of the means of accomplishment that pertain to it. An example template for such an aggregation is shown in Table 10.8.

Table 10.8 Example template for aggregating means of accomplishment to principles

Principle No. Principle Description Means of Accomplishment No. Means of Accomplishment Description Means of Accomplishment Rating (G/Y/R) Rating Rationale for Means of Accomplishment Aggregated Principle Rating (G/Y/R) Aggregated Rating Rationale
1 a
b
c
2 d
e
f
Etc.

In the current context, attainment of a green rating for each principle can be thought of as a type of operational objective of the organization, in that the principles generally have a shorter-term focus than strategic objectives. Since principles are a type of objective, it is possible to build a hierarchy of control loops for principles in the same way as was illustrated for objectives in Figure 10.6.

A generic example for the primary control loop addressing the achievement of a principle labeled as “X” is illustrated in Figure 10.12. The monitored variables consist of a rating for each means of accomplishment and an aggregated rating. The monitoring mechanisms include self evaluations and independent evaluations of the status of the principle based on the ratings for the means of accomplishment and the aggregated rating. In the event that the aggregation results in a rating other than green, the evaluators would provide recommendations for how to rectify that problem. The control mechanisms consist of the implementation of some or all of the recommendations resulting from the self and independent evaluations. The controlled variables are the means of accomplishment.

c10f012

Figure 10.12 Example generic primary control loop for achievement of internal control principles

Secondary and tertiary control loops should follow the approach discussed earlier in Sections 10.2 and 10.3, and will generally comprise both case-specific and generic controls. Case-specific controls are so-called because their design depends on the specifics of the control mechanisms identified in the primary loop. Generic controls, on the other hand, provide integrity to control processes in general. The candidate secondary and tertiary control loops listed in Table 10.4 would be applicable here as generic controls.

Figure 10.13 illustrates an instantiation of Figure 10.12 for the particular case of Principle 1: Demonstration of a commitment to integrity and ethical values by the oversight body and management. The means of accomplishment for this example are extracted from Table 10.6.

c10f013

Figure 10.13 Example primary control loop for demonstration of a commitment to integrity and ethical values

10.5 Summary of Observations

Internal controls should be derived from the organization's hierarchy of objectives and standards of operation and from considerations of the risk and opportunity drivers that affect the organization's ability to meet those objectives and standards. The risk and opportunity drivers are determined from the factors that most significantly affect aggregate risks and opportunities rather than just from individual risks and opportunities. The identification and evaluation of internal controls focus largely on protection of the assumptions and/or correction of the weaknesses that need to be addressed for the aggregate risks and opportunities to be effectively and efficiently controlled within the decision maker's risk tolerance and opportunity appetite.

Structures of control loops may be derived and implemented in a hierarchical manner. Such structures should contain a primary control loop and a nested series of subordinate control loops. Hierarchical control loops can be developed starting from an organization's strategic objectives to enhance the likelihood of meeting those objectives. They can also be developed from the principles of internal control espoused in various prior references such as the GAO Green Book to enhance the likelihood of meeting those principles.

The benefits of creating hierarchical control structures are the same as the benefits for creating hierarchical organizational management structures. First, they promote the definition of roles and responsibilities to ensure that all-important controls have a champion or owner as well as an oversight authority. This causes lower-level organizational units to be aware that they have control responsibilities that are intended to support the higher-level controls that exist at higher levels of the organization. Furthermore, hierarchical control structures have the advantage of being well-suited for integration with EROM structures, which also are hierarchical in nature.

Hierarchies of control loops are similar in concept to organizational hierarchies but do not have to mirror the organizational hierarchies. All that is necessary is that there be a mapping between the two so that the roles and responsibilities for formulating and implementing controls are clearly defined in terms of organizational entities.

The approach advocated here differs philosophically somewhat from the approach taken in the COSO framework for internal controls. The COSO framework posits that internal controls are an input to enterprise risk management (ERM) but that ERM is not necessarily an input to internal controls. The framework herein suggests a more bidirectional integration of internal controls and EROM, which is more suitable for organizations whose objectives are more technical than financial in nature.

Note

References

  1. Bolden, C. 2013. “Commercial Crew Development.” Wikipedia website (April). http://blogs.nasa.gov/bolden/2013/04/
  2. Canfield, A. 2016. “The Evolution of the NASA Commercial Crew Program Mission Assurance Process,” NASA Kennedy Space Center. https://ntrs.nasa.gov/archive/nasa/casi.ntrs.nasa.gov/20160006484.pdf
  3. Committee of Sponsoring Organizations of the Treadway Commission (COSO). 2004. “Enterprise Risk Management—Integrated Framework.”
  4. Committee of Sponsoring Organizations of the Treadway Commission (COSO). 2013. “Internal Control—Integrated Framework.”
  5. Government Accountability Office. (GAO). 2014. GAO-14-704G, The Green Book, Standards for Internal Control in the Federal Government. Washington, DC: Government Accountability Office.
  6. Kirkpatrick, P., and Vassberg, N. 2014. “The Evolution of the NASA Commercial Crew Program (CCP) Safety Process.” Proc. of 7th IAASS Conference: “Space Safety Is No Accident.”
  7. Leveson, N. 2011. Engineering a Safer World: System Thinking Applied to Safety. Cambridge, MA: The MIT Press.
  8. Leveson, N., et al., 2005. “Risk Analysis of NASA Independent Technical Authority.” Massachusetts Institute of Technology. website sunnydat.mit.edu.
  9. National Aeronautics and Space Administration (NASA). 2011. NASA/SP-2011-3422. “NASA Risk Management Handbook.” Washington, DC.
  10. Office of Management and Budget (OMB). 2016. Circular A-123. “Management's Responsibility for Enterprise Risk Management and Internal Control.” Washington, DC.
  11. Smith, M., and Erwin, J. 2005. “Role & Responsibility Charting (RACI),” PMForum.org.
  12. VanDoren, V. 2014. “Fundamentals of Cascade Control.” Control Engineering. Website http://www.controleng.com/single-article/fundamentals-of-cascade-control/bcedad6518aec409f583ba6bc9b72854.html
..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset
18.227.21.70