2.2.1 Activities within Each Management Level

At the executive level, the processes of strategic planning, strategic plan implementation, and strategic performance evaluation are guided by information obtained from both external and internal sources, as shown in Figure 2.3. The information to be gleaned from external sources includes:

• Mission priorities, programs/projects, schedules, and budgets that are mandated by external stakeholders and funding authorities, such as Congress and the US president in the case of federal agencies
• Supply constraints such as the availability of suppliers, parts, and materials
• Marketplace constraints such as inflation rates and competition from other entities, both domestic and foreign
• Political constraints, such as the prospects for changes in the federal administration, the makeup of Congress, restrictions on certain foreign entities, or the leadership of nongovernment funding agencies
• Legal constraints, such as new enactments with new requirements or threats of litigation
• The emergence of new technology that may open opportunities for undertaking new objectives or achieving faster progress toward current objectives, or conversely pose new threats (e.g., cyber-security)

In addition, information is transferred from the executive level to entities external to or independent from the TRIO enterprise management structure, such as (for federal agencies) the GAO, the OMB, inspectors general, and Congress, in the form of presentations and reports. The scope and contents of information provided to OMB has to comply with the requirements of GPRAMA as detailed in various OMB circulars.

Information to be received from internal sources (programmatic and institutional/technical levels) includes:

• The status of risks and opportunities for programs/projects, including safety concerns, technical performance concerns, cost concerns, and schedule concerns
• The status of risks and opportunities at the institutional/technical level, including workforce concerns, concerns with facilities and equipment, IT concerns, and security concerns
• Identification and evaluation of risks and opportunities that cut across programs, projects, and institutional/technical entities
• The status of concerns within the programmatic and institutional/technical levels that have evolved from risks to problems, and the status of corrective actions

Correspondingly, information is transferred from the executive level to the programmatic and institutional/technical levels via the strategic plan, and associated back-up material, including in particular the specifications for the agency's portfolio of programs, projects, institutional initiatives, research and development initiatives, resource expectations, schedules and budgets, and so on.

The activities and transfer of information at the programmatic or program directorate level parallel the activities and transfer of information at the executive level, but with the following differences as shown in Figure 2.4:

• The top objectives are programmatic and, for the most part, are received from the executive level as part of its strategic planning and plan implementation activities.
• The results from the programmatic planning, implementation, and performance evaluation activities are presented to the various governing councils within the TRIO enterprise, which may include (for example) a strategic management council, an executive council, a program management council, and/or a mission support council.
• The results from the programmatic performance evaluation also provide input to portfolio performance reviews.
• Implementation of the programmatic planning activity includes feedback to and from other program directorates, particularly regarding concerns that cut across program directorates.

By and large, the program directorates operate as enterprises, so from a practical point of view, the principles of EROM apply to them as well as to the executive level.

The same is true for the technical centers or directorates,¹ as shown in Figure 2.5. The activities and transfer of information at the center level parallel the activities at the program directorate level, except that the top objectives concern institutional and technical capability development as well as support of the programs/projects. These top objectives require the technical centers to concentrate, in their planning processes, upon how to achieve an efficacious balance between services provided directly by them versus services acquired from other entities such as commercial companies, universities, and other agencies.

2.2.2 Roles and Responsibilities within and between Each Management Level

Ensuring that managerial roles and responsibilities are clearly defined and that there are no gaps in the assignment of these roles and responsibilities is a major element of enterprise risk management and internal controls. Table 2.1 presents a representative list of roles and responsibilities at the executive, program directorate, and technical directorate levels for a typical TRIO enterprise. The entries in the table were adapted from NASA (2014a) (Table D-1), and they elaborate further on the information conveyed in Figures 2.3 through 2.5.²

2.3.1 Organizational Planning and Plan Implementation

The manner in which EROM assists management at all three levels in developing a responsive and achievable plan is illustrated in Figure 2.6. Following is a brief summary of the activities depicted in this figure:

Management activities that provide input to the EROM process include:

• Understand and comply with external constraints such as mandated missions and programs, mandated budgets, the availability of suppliers and parts or materials, and legal realities.
• Identify alternative objectives hierarchies that comply with the external constraints and have the potential for achieving the organization's mission in all time frames.

EROM activities that provide input to the management activity of selecting among alternative objectives and preparing the organizational plan include:

• Characterize and understand all relevant historical experience pertaining to failures, successes, precursors, anomalies, unexpected benefits, and lessons learned.
• Identify risks and opportunities for each alternative set of objectives based on the historical record and expert judgment.
• From past experience and current risk/opportunity leading indicators, assess the state of risks/opportunities as they pertain to the likelihood of achieving each objective.
• Risk-inform the selection and application of internal controls.

2.3.2 Evaluation of Organizational Performance and Replanning

The evaluation of performance at the various management levels also involves close coordination between management activities and EROM activities. From an EROM perspective, the activities that support performance evaluation are similar to the activities that support organizational planning in the sense that both involve the identification and evaluation of risks and opportunities. As discussed in Section 1.2.2, the key difference is in the level of maturity that exists in the definition of risks and opportunities.

The manner in which EROM assists management in evaluating organizational performance is illustrated in Figure 2.7. Following is a brief summary of the activities depicted in that figure:

Management activities that provide input to the EROM process include:

• Track progress on individual programs, projects, institutional initiatives, and other activities in the portfolio with respect to meeting the mid- and short-term objectives in the organizational objectives hierarchy.
• Conduct a portfolio performance review (PPR) at periodic intervals to assess overall adherence to the performance plan and to identify and evaluate cross-cutting issues.

EROM activities that provide input to the management activity of conducting the portfolio performance review include:

• Track leading indicators that pertain to organizational risks and opportunities. (Note that executive-level risks and opportunities generally emanate from external sources such as political, economic, or regulatory changes, whereas risks and opportunities at lower management units generally emanate from internal sources such as the depletion of reserves and margins in any of the mission execution domains: safety, technical performance, schedule, and cost.)
• From the current values of the leading indicators, assess the significance of the risks and opportunities at each level in the organizational objectives hierarchy.

EROM activities that provide input to the management activity of evaluating organizational performance include:

• Identify and track internal performance measures and internal/external leading indicators of risks and opportunities that pertain to the mid- and short-term organizational objectives.
• From the current values of the performance measures and leading indicators and their observed trends, assess the state of risks and opportunities as they pertain to the likelihood of achieving the top organizational objectives.
• When risks are of concern, or when opportunities are attractive, perform an analysis to suggest options that may be pursued to mitigate risks or pursue opportunities and identify associated internal controls.

With these inputs in hand, management has a solid basis for determining whether the organization's objectives are being achieved and whether there are imposing reasons (either positive or negative) for amending or changing some of the objectives and/or portfolio elements. The organization also is in a better position to prepare performance reports and presentations of the type required by the external stakeholders and funding agencies.

2.3.3 Alignment with Management-Level Roles and Responsibilities

Table 2.2 provides a more detailed itemization of EROM activities to support the various management levels of a TRIO enterprise consistent with the roles and responsibilities listed in Table 2.1. The entries in the table elaborate further on the information conveyed in Figures 2.6 and 2.7.

2.4.1 Nature of the Strategic Objectives That Require Extended Partnerships

Large not-for-profit and government TRIO organizations tend to have a diversity of strategic objectives that go beyond technical and scientific accomplishments related to the prime mission to geopolitical, macroeconomic, and societal objectives that require extensive collaboration. Following, for example, are several strategic objectives (S.O.s) from NASA's strategic plan that fall into this category (emphasis added to highlight the point):

1. [S.O. 1.1] Expand human presence into the solar system and to the surface of Mars to advance exploration, science, innovation, benefits to humanity, and international collaboration.
2. [S.O. 1.2] Conduct research on the International Space Station (ISS) to enable future space exploration, facilitate a commercial space economy, and advance the fundamental biological and physical sciences for the benefit of humanity.
3. [S.O. 1.3] Facilitate and utilize US commercial capabilities to deliver cargo and crew to space.
4. [S.O. 1.7] Transform NASA missions and advance the Nation's capabilities by maturing crosscutting and innovative technologies.
5. [S.O. 2.4] Advance the Nation's STEM education and workforce pipeline by working collaboratively with other agencies to engage students, teachers, and faculty in NASA's missions and unique assets.

Objectives such as these require TRIO enterprises to work collaboratively with other US agencies, foreign agencies, commercial entities, and educational entities. Most of the collaboration takes place within projects, programs, and special activities (such as new technology development) that are designed to satisfy the strategic objectives of the managing organization.

2.4.2 The Challenges of Conducting EROM across Extended Partnerships

Implementing an effective EROM process within an enterprise that depends on extended partnerships can be challenging. For example, according to a deputy director for US Department of Defense's National Geospatial-Intelligence Agency (Holzer 2006), writing about the practice of EROM across extended partnerships: “Culture resistance to change and unwillingness to share information viewed as negative prevail. There is additional complexity convincing people to adopt a process that is part of the bigger organization and sharing information regarding their ability to achieve program objectives.”

In general, the following attitudinal and operational perspectives are needed to accomplish a satisfactory implementation of EROM when extended partnerships are involved (Holzer 2006; Perera 2002):

• Managers within each of the partners need to be convinced that making risk known to all participants in the extended partnership will be positively recognized and at times rewarded with an allocation of risk mitigation funds.
• Partners whose components or systems are being integrated with those of other partners need to be convinced that it is to their benefit to collaboratively and cooperatively manage risks evolving from the integrated relationships.
• When joining enterprises managed by distinctly different organizations to create an extended partnership, diverse leaderships, objectives, motivations, and other cultural views (and ways of doing risk management) need to be melded in accordance with proprietary, security, foreign dissemination (ITAR), and other considerations.

According to various sources, the single most important factor for achieving buy-in across an extended partnership is for senior leaders of each partnering organization, especially at the top level, to repeatedly voice their support and enforce accountability for an integrated risk and opportunity management process across the partnership.³

2.5.1 OMB Circular A-11 and GPRAMA (Government Performance, Results, and Budgeting)

The July 2016 release of OMB Circular A-11 (OMB 2016a) has several new sections devoted to enterprise risk management. Following are three relevant quotations from these sections:

• Section 270.24 states that “Enterprise risk management (ERM) is an effective agency-wide approach to addressing the full spectrum of the organization's significant risks by understanding the combined impact of risks as an interrelated portfolio, rather than addressing risks only within silos. ERM provides an enterprise-wide, strategically-aligned portfolio view of organizational challenges that provides better insight about how to most effectively prioritize and manage risks to mission delivery.”
• Section 270.25 states that “ERM is a strategic discipline that can help agencies to properly identify and manage risks to performance, especially those risks related to achieving strategic objectives. An organizational view of risk positions allows the agency to quickly gauge which risks are directly aligned to achieving strategic objectives, and which have the highest probability of impacting mission When well executed, ERM improves agency capacity to prioritize efforts, optimize resources, and assess changes in the environment.”
• Section 270.26 states that “While agencies are not required to have a CRO [chief risk officer] or enterprise risk management function, they are expected to manage risks to mission, goals, and objectives of the agency. Where applicable, a CRO or other person designated with these responsibilities may serve as a strategic advisor to the COO [chief operating officer] and other staff on the integration of risk management practices into day-to-day business operations and decision-making.”

GPRAMA and OMB Circular A-11 also talk about leading indicators that enable the agency to show that it is on track with respect to meeting its goals and objectives, and in cases where it is not on track, to understand the causes of difficulty and how they can be corrected. The GPRAMA legislation contains the following provisions that are relevant to this discussion:

• In Paragraph 306: “The head of each agency shall make available on the public website of the agency a strategic plan [that] shall contain…an identification of those key factors external to the agency and beyond its control that could significantly affect the achievement of the general goals and objectives.”
• In Paragraph 1121: “Use…performance information to achieve agency priority goals…[and] for agency priority goals at greatest risk of not meeting the planned level of performance, identify prospects and strategies for performance improvement, including any needed changes to agency program activities, regulations, policies, or other activities.

Amplification provided in OMB Circular A-11 includes the following observations:

• In Section 200.21, “Other indicators [are] indicators not used in a performance goal or Agency Priority Goal statement but are used to interpret agency progress or identify external factors that might affect that progress.”
• Also in Section 200.21, “Outcome [indicators are] a type of measure that indicates progress against achieving the intended result of a program [and that] indicates changes in conditions that the government is trying to influence.”

The indicators referred to here may be inferred to be risk leading indicators because they focus on factors that impede progress toward future results.

In addition, OMB Circular A-11 talks about the desirability of pursuing opportunity. Quoting from the Executive Summary:

• “The Administration expects agencies to set a limited number of ambitious goals that encourage innovation and adoption of evidence-based strategies. Agency leaders at all levels of the organization are accountable for choosing goals and indicators wisely and for setting ambitious, yet realistic targets. Wise selection of goals and indicators reflects careful analysis of the characteristics of the problems and opportunities an agency seeks to influence to advance its mission.”
• “As important as it is to sustain a strong performance culture through the practices described in the guidance, it is equally important to have reliable and effective processes which support continuous improvement and opportunities for capacity building.”

The principal ways in which EROM helps ensure compliance with GPRAMA and with the OMB Circular is through the emphasis it provides in having a robust process for selecting goals and objectives both long-term and short-term, in considering risk and opportunity leading indicators to evaluate the likelihood of success, and in placing opportunity pursuit on an equal basis with risk control. These facets of EROM are apparent from Figures 2.6 and 2.7.

2.5.2 EROM and Internal Controls from the Viewpoint of Federal Regulations and Guidance

Under the new federal regulations and related guidance, the activities involved in conducting EROM are intimately related to and mutually supportive of the activities involved in specifying, implementing, and maintaining internal controls.

According to Circular A-11 (OMB 2016a), “Internal controls are the organization, policies, and procedures that [an] agency uses to reasonably ensure that:

• Programs achieve their intended results.
• Resources used are consistent with agency mission.
• Programs and resources are protected from waste, fraud, and mismanagement.
• Laws and regulations are followed.
• Reliable and timely information is obtained, maintained, reported and used for decision making.”

Within the context of EROM, internal controls can be viewed as processes that the organization decides to implement to provide defense-in-depth against risks and to promote successful achievement of its strategic goals and objectives. The overall set of responses to risks and opportunities may include additions or modifications to the design, fabrication, assembly, testing, and operation of a system to mitigate risks and exploit opportunities within the framework discussed earlier. Internal controls focus on processes, procedures, and protocols that make it possible for the overall set of responses to succeed.

According to COSO (2004), “Internal control is encompassed within and an integral part of enterprise risk management. Enterprise risk management is broader than internal control, expanding and elaborating on internal control to form a more robust conceptualization focusing more fully on risk.”

Some typical examples of internal controls are cited in the last previous version of OMB Circular A-123, as follows (OMB 2004):

• “Policies and procedures;
• Management objectives (clearly written and communicated throughout the agency);
• Planning and reporting systems;
• Analytical review and analysis;
• Segregation of duties (separate personnel with authority to authorize a transaction, process the transaction, and review the transaction);
• Physical controls over assets (limited access to inventories or equipment);
• Proper authorization;
• Appropriate documentation and access to that documentation.”

These controls tend to focus heavily on protecting programs and resources from waste, fraud, and mismanagement and on protecting entities from legal liability. In addition to these, the identification, tracking, and analysis of risk leading indicators is another type of internal control that addresses an organization's strategic risk and helps the organization to achieve its mission. This type of internal control is addressed more fully in the most recent issuances of Circulars A-123 and A-11.

In the realm of strategic planning, there are risks pertaining to the setting of objectives (such as failing to have reliable information from external entities), and there are controls to manage those risks (such as ensuring that reliable information is obtained and provided to those responsible for setting the objectives). Failure to have the correct information may also affect the ability to conduct effective risk management once the objectives have been decided on. There should be controls to address these risks as well (Marks 2013).

In determining whether a particular control should be established, the risk of failure and the significance of the opportunity are considered along with the related costs (COSO 2004). For example, it may not be cost-effective for a TRIO enterprise to install sophisticated inventory controls to monitor levels of raw material if the cost of the raw material used in a production process is low, the material is not perishable, ready supply sources exist, and storage space is readily available. Excessive controls that do not address significant risks or opportunities are likely to be costly and unproductive. In addition, they may actually increase risk due to the added burden of having to implement an unnecessary control.

2.5.3 OMB Circular A-123 (Management's Responsibility for ERM and Internal Control) and the Required Statement of Assurance

OMB Circular A-123 (OMB 2016b) concerns management's responsibility for integrating internal control with enterprise risk management. The memorandum introducing the new circular to the various government agencies states that the intent of the changes from the previous version is “to modernize existing efforts by requiring agencies to implement an Enterprise Risk Management (ERM) capability coordinated with the strategic planning and strategic review process established by GPRAMA, and the internal control processes required by FMFIA [Federal Managers Financial Integrity Act] and Government Accountability Office (GAO)'s Green Book.” The tenor of the new circular is intentionally at a high level to allow each agency the latitude of developing approaches that are applicable to it.

OMB (2016b) views internal controls as being contained within enterprise risk management, and the latter as being contained within governance (see Figure 2.8). As stated in OMB (2016b): “Most agencies should build their capabilities, first to conduct more effective risk management, then to implement ERM, rating those risks in terms of impact, and finally building internal controls to monitor and assess the risk developments at various time points.” Furthermore: “To provide governance for the risk management function, agencies may use a Risk Management Council (RMC) to oversee the establishment of the Agency's risk profile, regular assessment of risk, and development of appropriate risk response.”

The broad governance structure of the federal government is defined through a variety of sources, and in particular, according to OMB (2016b), the “core governance processes are defined by…OMB budget guidance, such as OMB Circular No. A-11, which defines the processes by which the Executive Branch develops and executes Strategic Plans, compiles the President's Budget request, assembles Congressional Budget Justifications, conducts performance reviews, and issues Annual Performance Plans and Annual Program Performance Reports.”

According to OMB (2016b), each federal agency is required to submit a statement of assurance (SoA) that “represents the agency head's informed judgment as to the overall adequacy and effectiveness of internal control within the agency.” According to NASA (2014b), “GAO and OMB are seeking to clarify existing guidance on internal controls…In the past, this review has been largely focused on financial [matters]…The clarifying guidance also seeks to more constructively focus on the concepts of integrated informed risk/risk-based system of internal controls that is not new but previously overshadowed by the financial focus.”

OMB (2016b) emphasizes the importance of having appropriate enterprise risk management processes and systems to identify challenges early, to bring them to the attention of agency leadership, and to develop solutions. It synthesizes existing EROM material mainly from COSO and from the UK Orange Book (2004) while relying on the GAO Green Book (GAO 2014a) as the primary source for the principles relating to internal controls. As described earlier in Section 1.1.4 of this book, COSO provides an overarching EROM framework that is particularly relevant to private enterprise. The Orange Book, correspondingly, offers an EROM framework that is relevant to federal agencies especially in the United Kingdom. The technical references to the Orange book in the new OMB Circular A-123 mainly concern the derivation of risk profiles and the development of models of EROM defined largely in terms of the relationships between different entities.

The following statements are direct quotations (with emphasis added by the author) highlighting some of the new requirements placed by Circular A-123 on federal agencies:

• “[The circular] requires agencies to integrate risk management and internal control functions.”
• “Federal leaders and managers are responsible…for implementing management practices that can effectively identify, assess, mitigate, and report on risks.”
• “Annually, agencies must develop a risk profile coordinated with their annual strategic reviews.”
• “[Risk profiles should] identify risks arising from mission and mission-support operations.”
• “A portfolio view of risk [should provide] insight into all areas of organizational exposures to risk, such as reputational, programmatic, performance, financial, information technology, acquisitions, human capital, etc.”
• “For those objectives for which formal internal control activities have been identified as part of the Risk Profile, assurances on internal control processes must be presented in its Annual Financial Report (AFR) or Annual Performance Report (APR), along with a report on identified material weaknesses and corrective actions.”
• “Agencies should develop a “maturity model approach” to the adoption of an ERM framework.”
• “For FY 2016, Agencies are encouraged to develop an approach to implement ERM. For FY 2017 and thereafter Agencies must continuously build risk identification capabilities into the framework to identify new or emerging risks, and/or changes in existing risks.”

2.5.4 Example Risk Profile from OMB Circular A-123

One of the principles in Circular A-123 pertaining to the development of the risk profile is that the assessment should “ensure that there is a clearly structured process in which both likelihood and impact are considered for each risk.” Table 1 of OMB (2016b) provides an example of a risk profile that specifically reports likelihood and impact as separate items for both inherent risk (the risk before instituting internal controls) and residual risk (the risk after instituting internal controls). The example is reproduced here in Table 2.3.

As stated in OMB (2016b), “While agencies can design their own appropriate categories, for the purposes of this guidance the following illustrative definitions can be used.” For impact:

• “High: the impact could preclude or highly impair the entity's ability to achieve one or more of its objectives or performance goals;
• “Medium: the impact could significantly affect the entity's ability to achieve one or more of its objectives or performance goals; and
• “Low: the impact could not significantly affect the entity's ability to achieve each of its objectives or performance goals;”

and for likelihood:

• “High: the risk is very likely or reasonably expected to occur;
• “Medium: the risk is more likely to occur than unlikely; and
• “Low: the risk is unlikely to occur.”

An alternative suggested ranking process more suitable for TRIO enterprises will be presented and discussed in Section 3.6.3.