Survey Respondent Profile

A broad range of industries were represented in the survey as shown in Exhibit 22.1: 32 percent in financial services, 18 percent in the utility sector, 9 percent in telecommunications, 9 percent in the public sector, 7 percent in energy, 5 percent in manufacturing, 5 percent in health care, and 15 percent in other industries. See Appendix 22.B for a list of companies that responded to the survey and gave us permission to be identified. Since the survey was given through the CBoC, most respondents were from Canada but 16 percent were from the United States. Although 78 percent of the companies’ operations were primarily in the United States and Canada, 28 percent of the respondents worked for companies that had operations in at least one international country (and almost all had global operations). Most organizations that participated in the survey were large and the average size was approximately $27 billion in total assets and 18,000 employees. The largest participating organization was General Motors. However, a few small businesses participated in the survey: approximately 10 percent of the survey respondents had fewer than 100 employees but only one organization had assets less than $1 million.

Exhibit 22.2 lists the numbers of years of experience that survey respondents and companies have had with ERM. As shown, all respondents had some experience, and 95 percent listed risk management as their primary area of expertise. The mean ERM experience was 5.3 years and approximately 40 percent of the respondents have more than 5 years of experience. Only one respondent had less than one year of experience with ERM and 11 percent had less than two years of experience. The respondents had more years of ERM experience on average than their organizations (5.3 years versus 3.8 years). Most companies that responded have implemented ERM to a certain extent. Approximately 88 percent of companies had more than one year of experience and more than 60 percent had at least three years of experience. These results are consistent with other surveys indicating that companies are moving toward more advanced stages of ERM as external stakeholders, rating agencies, and analysts expect more information on risk management techniques being employed.¹¹

Most survey respondents held high positions within the organization: more than one-half (52.3 percent) held positions at the chief risk officer level or higher. The largest group in the survey held the title of director (31.8 percent) while 9.1 percent were chief officers (not risk). Most respondents reported to top officials of the organization: 31 percent to the chief financial officer and 26.2 percent to the chief executive officer.¹² It is interesting to note that 24 percent stated they also reported functionally to the audit committee.

Exhibit 22.3 lists the most frequently cited benefits by executive management of implementing ERM. Respondents were allowed to list multiple benefits. As shown, the most cited benefit is “Better understanding and management of risk (including an integrated view).”¹³ This benefit, cited by 44.7 percent of respondents, shows a high level of acceptance of ERM and suggests that companies genuinely understand the importance of this advanced risk process. The second most cited reason (18.4 percent), “Improve corporate governance or meet board requirements,” reflects recent regulatory changes and the increased emphasis on corporate governance. Another survey by Gates (2006) has found a higher percent (66 percent) listing this benefit.¹⁴ Given that 84 percent of the organizations in our study are Canadian and are less likely to be required to comply with Sarbanes-Oxley (SOX), the second-place ranking is not surprising.¹⁵ It is interesting to note that 10.5 percent listed improving their credit rating as a benefit of ERM. We expect this percentage to increase over time given that ratings agencies are now including ERM as part of their ratings process for nonfinancials.¹⁶

ERM Tools and Techniques Used by Respondents

Do risk executives follow COSO’s ERM recommended tools and techniques? Exhibit 22.4 summarizes the survey responses. Surprisingly, 19 organizations (48.7 percent) responded they seldom do this, 20.5 percent responded “sometimes,” and only 30.8 percent responded “to a large extent.” No organization responded “as much as possible.” While COSO is the most read resource (see later discussion on this), it does not appear to be the most useful for actual practice at this time. Anecdotal input from informal surveys and roundtables indicate that COSO is written in a style that is hard to read and to absorb. It is our belief that many readers give up partway through and therefore do not refer to COSO or use its ideas in practice. However, this means that there is an important opportunity for COSO to be rewritten in the future. Protiviti’s (2006) “Guide to Enterprise Risk Management: Frequently Asked Questions” seems to have garnered greater readership and to be an easier document to read and understand.

So how useful are other sources of best practices and methodology for ERM? Exhibit 22.5 answers this question for the following sources: COSO, public accounting firms and consultants, professional associations (RIMS, PRIMIA, SOA, etc.), newspapers and magazines, academic journals and papers, and literature in general. Response choices were: 1=seldom; 2=fair/occasional; 3=good/frequent, and 4=as much as possible. As shown in Exhibit 22.5, risk executives rated knowledge of the literature as the highest source of guidance on ERM practices and methodology (mean rating of 3.08), followed by professional associations as the next most useful source of information (mean rating of 2.52). Consistent with Exhibit 22.4, COSO received the lowest rating of 1.81.

How useful are consultants to the implementation of ERM? Fifty-nine percent of the organizations have used consultants to help with their journey in ERM. In response to the question “Do you feel you have learned more from reading and researching ERM than from consultants?” it appears respondents find the literature more helpful: 53 percent responded “yes,” 39 percent responded “somewhat,” and 8 percent responded “no.” Respondents were allowed to comment regarding their responses. The following comments illustrate three of the key concerns executives face with consultants:

1. Consultants have no choice but to provide generic/academic frameworks and tools. Only in-house management can implement a true ERM approach for their own company because they know their business, processes, culture, and just what makes sense for them that no outside party can truly know. It becomes inefficient to educate an outside party on your business just so they can try to tell you what you should be doing (from generic models) to manage it better.
2. Consultants generally advocate a single perspective—often a COSO view—which we find too restrictive and compliance-based. Some consultants advocate the use of Basel but it is not a good fit for our industry. An ERM program needs to be developed from within. We have used the Australian Standard 4360 to help build our program.
3. Some articles (if current) are sometimes more pragmatic and “out of the box” versus consultants. Consultants seem to have capabilities around risk assessment, but less so for robust ERM framework/implementation efforts.

Although it is clear that risk executives as a group find ERM literature more helpful, several respondents indicated the benefits of consultants, too:

• My belief is that consultants are helpful in the “getting started” phase and also for specific tasks, such as facilitating a risk profiling process with an executive group.
• Consultants can be useful but I want to know the theory and practice myself so that I can direct and check the recommendations of consultants.
• The consultants were useful in the implementation of what we had decided we wanted as a framework. However, they provided good value in benchmarking best practices that we would not have been able to do.

And one must be careful with the literature, as one survey respondent points out,

The problem is sorting out the good readings from the bad (or even harmful).

We also investigated the relationship between risk executives experience and their familiarity with ERM literature using the categories shown in Exhibit 22.5. Experience was measured as the number of years the respondent had with ERM. Although we find no significant relationship between risk executives’ experience and their ratings on the benefits of COSO and other major sources of ERM information, we do find that more experienced risk executives had a greater knowledge of the literature than their less experienced counterparts (Pearson correlation coefficient of 51 percent; significant at the 1 percent level). We discuss the relation between experience and the most frequently read literature in more detail in the next section.

Risk executives in higher positions had read significantly more than those in lower positions (Pearson correlation coefficient of 28 percent; significant at the 10 percent level).¹⁷ We also found that risk executives in higher positions rated academic papers less useful (Pearson correlation coefficient of –19 percent but insignificant at the 10 percent level). Although the result is not significant at conventional levels, it is worth noting and in contrast to the finding of almost no relationship between years of experience and usefulness of academic papers. Given that few academic papers have been published on ERM, one should not draw any strong conclusions from this result other than the indication that there is a crucial need for academics to publish more useful research on ERM.

Most Frequently Read Literature on ERM

Now to the main objective of our study: to uncover the most useful literature read by risk executives. As discussed in the previous section, we asked respondents to rate each reading by answering the following two questions: (1) Did you read this book/research paper/article and if so to what extent?; and (2) in terms of adding value to your knowledge of ERM, how would you rate this book/research paper/article according to methodologies, tools, techniques, and leading practices for ERM? (Note: For discussion purposes, we refer to the Question 1 response as “read” and the Question 2 response as “value.”)

We classified the 88 readings according to articles (24 total, which includes surveys, academic studies, and practitioner articles), books (32 total), and research reports (32 total). Exhibit 22.6 summarizes the mean ratings of the readings for all publications and by type (i.e., articles, books, and research reports). Panel A summarizes the “read” and “value” ratings and Panel B analyzes the ratings based on the respondents experience with ERM. In Panel B, risk executives with five years or more experience were classified as having “high experience” and those with less than five years experience were classified as “low experience.” (Note: The mean level of experience of all risk executives was 5.3 years.) As shown in Panel A, the mean ratings for “read” and “value” do not differ greatly according to publication type. However, in Panel B, risk executives with greater experience were more familiar with all publication types (difference of means t-test significant at the one percent level in all groups). There was no significant difference in the “value” rating based on experience.

To select the “top readings” individually, we first ranked the readings by type (i.e., articles, books, and research reports) using a weighting scheme based on the responses to the two questions. We then sorted the ranked categories into quartiles and the readings, which were ranked in the top quartiles based on both questions that were first considered as “top readings.” Only literature rated by at least six respondents was considered in the final rankings.¹⁸ A few articles with second quartile rankings still made the top 10 lists. The results of our “Top 10” readings are presented in Exhibits 22.7, 22.8, and 22.9 for the articles, books, and research reports, respectively. Our survey participants may not represent all ERM executives’ familiarity with the literature, but, to our knowledge, we present the first survey evidence on this important topic. Anyone wishing to learn more about ERM should consider placing these publications on their “must read” list.

Exhibit 22.7 lists the top 10 articles on ERM sorted according to the year of publication. As mentioned earlier, we include surveys, academic studies, and practitioner articles in this category. Although not indicated in the table, the highest ranked study in this category is “Risk Management Reports” by H. Felix Kloman (later Beaumont Vance), followed by “Enterprise Risk Management at Hydro One Inc.” by Fraser, Quail, and Kirienko (2001).¹⁹

The top 10 books on ERM are listed in Exhibit 22.8.²⁰ The books receiving the highest overall rating are 20 Questions Directors Should Ask about Risk by Lindsay, Fraser, Goodfellow, and Toledano (2006) and the COSO publication, “Enterprise Risk Management: Integrated Framework: Executive Summary” (2004). This COSO publication was the most well read in our survey (mean “read” rating of 4.13; read by 74 percent of survey respondents) but received a mean “value” rating of 2.45, which can be viewed as an average rating. This is consistent with our findings discussed earlier regarding the COSO publications.

This table reports summary ratings of ERM literature based on the following two survey questions: (1) Did you read this book/research paper/article and if so to what extent? Response choices were: 1=never heard of it, 2=heard of it, but not really read it, 3=read less than 10% of it, 4=read between 10%–80%, and 5=read more than 80%), and (2) in terms of adding value to your knowledge of ERM, how would you rate this book/research paper/article according to methodologies, tools, techniques and leading practices for ERM? Response choices were: 1=not really relevant to ERM, 2=some value but not a lot, 3=reasonably useful, 4=very good in ERM, and 5=a must read for ERM. The Question 1 and Question 2 responses are reported in this table as “Read” and “Value,” respectively. Panel B reports the results of ratings based on the respondents experience with ERM. Respondents with 5 years or more were classified as having “high experience” and those with less than 5 years were classified as “low experience.” The panel also presents univariate tests of the differences in mean values between ratings for the high and low experience groups. The t-statistic provides a test of the null hypothesis that the mean value does not differ between the two groups. Significance levels are indicated as follows: ***1%, **5%, *10%.

Exhibit 22.9 lists the top 11 research reports. Eleven reports are listed due to a tie for 10th place. Three research reports received significantly higher ratings than other reports and are as follows (listed in order of ranking): “Risk Management” by AS/NZS 4360 (1995, 1999, and 2005), “Guide to Enterprise Risk Management: Frequently Asked Questions” by Protiviti (2006), and “ERM: Inside and Out” by Thiessen (2005).

Are there other useful readings we omitted from our study? We asked respondents to identify literature they found useful in early stages and advanced stages of ERM that we had omitted from our list. The most frequently mentioned publications are listed in Exhibit 22.10. Panel A lists studies useful in early stages and Panel B lists ones useful in more advanced stages. Interestingly, respondents indicated that some of the best literature they have read does not necessarily mention ERM, but simply addresses various aspects of risk. The variety of risk literature fits with the fact that the respondents come from diverse lines of businesses, industries, and corporate structures, not to mention representing a large range of individual interests. It should be noted that only one publication was mentioned by more than one respondent (i.e., Black Swan); all others were only mentioned once. This supported the validity of our original survey lists that there were no major omissions. The Black Swan was omitted from our survey list as it was published in April 2007 during the compilation of our survey list.

Critical Areas of Need

Answers provided to open-ended questions in the survey suggest that there is a critical need for more detailed “real-world” applications on ERM. In response to the question, “What problems/challenges have you encountered in implementing ERM that were not addressed in the literature?” the following quotes by risk executives summarize key areas of need:

• In addition, virtually all literature is silent on how to deal with the myriad cultural, logistical, historical challenges that exist and are unique to all organizations. These (and other) challenges create significant (and sometimes insurmountable) barriers that must be addressed if an organization hopes to manage risk on an integrated basis.
• Many of the articles describe what the process should look like and how it should function but there are few that provide details of how to get to that step. Many of the articles use great overarching statements that seem very much like motherhood statements. There was a distinct lack of information on how to bring all the silos together—other than to say that a common reporting system and language are important. It was difficult to find true life examples of how the information was gathered and presented to show a greater risk picture.
• The impact of corporate culture on ERM implementation and practices is not well addressed in the literature.

Key Findings of Our Survey

To summarize the most important results of our survey, we identify the following five findings. Our results help illuminate areas of need in the practice of ERM. We hope that our results are useful to practitioners wanting to learn more about enterprise risk management and also to academics interested in conducting research in this crucial area.

1. Surprisingly, COSO was not being considered and used as the key source of information and guidance.
2. Challenges remain for new implementers, especially as to specific guidance on what to do in their cultural context.
3. Much more work is needed in the areas of research and case studies so that risk executives can learn from the experiences of others who have successfully implemented ERM. More specifically, risk executives are looking for more practical “how to’s,” sharing of experiences, impacts of different corporate culture, and best practices at the different stages of ERM implementation. This is an excellent opportunity for academics to closely collaborate with practitioners to conduct research in these key areas of need. (Note: What was read in the top 10 articles, books, and research was mostly about the “how to” aspects of ERM.)
4. Despite the wealth of practical experience of survey respondents, most of whom are from large companies, there clearly remain many areas to explore and discuss before a common understanding or methodology for ERM could be considered to be in place.
5. Experienced risk executives are more familiar with the literature and also find publications about “risk in general” very useful at early and advanced stages of enterprise risk management implementation.