We analyzed 124 reports—24 from South African companies and 100 from companies in other countries. Reports were selected if they were:

1. Self-declared to be an integrated report
2. Published by a company of any size listed on a stock exchange
3. Published in 2013 (for the 2012 fiscal year)
4. Published in English
5. Publicly available
6. Available for download in a PDF format

Companies producing the report could be from any geographic region and industry sector, and they could use any reporting framework. Reports meeting our criteria were sourced from Global Reporting Initiative's (GRI) Sustainability Disclosure Database¹ on October 17, 2013.

The analysis team used the “Consultation Draft of The International <IR> Framework”² (Consultation Draft) issued in July 2013 as the basis for developing the evaluation methodology, recognizing that all of the reports analyzed were published prior to the release of “The International <IR> Framework”³ (<IR> Framework) in December 2013. Twenty factors were used for scoring reports. Seven of these came from the Consultation Draft's “Content Elements.” Even though the Consultation Draft was used for our report assessment, the Content Element names are the same as those later used in the <IR> Framework.

• Organizational overview and external environment
• Governance
• Business model
• Risks and opportunities
• Strategy and resource allocation
• Performance
• Outlook

The Consultation Draft's “Six Capitals” (financial, manufactured, natural, intellectual, human, and social and relationship) were also used as scoring factors in the analysis.

In addition to the 13 factors from the Consultation Draft, we added the following seven “Special Factors” that we created:

• Identifies material risks
• Explains how material risks are handled/mitigated
• Uses a “Materiality Matrix” in the report to present the risks
• Demonstrates or explains stakeholder engagement
• Demonstrates or explains connectivity of information
• Website content supports/communicates integrated report content
• Letter from CEO or CSO (Chief Sustainability Officer) addresses organization's sustainability

Our category of “Special Factors” was similar to the Consultation Draft's “Guiding Principles,” with several exceptions. We felt that some principles (“Conciseness,” “Reliability and completeness,” and “Consistency and comparability”) would be difficult to assess in a reasonably objective manner. We also felt that “Strategic focus and future orientation” was largely repeated in the content elements of “Risks and opportunities,” “Strategy and resource allocation,” and “Outlook.” We strictly followed the Consultation Draft's six capitals, and all of its Content Elements except for “Basis of preparation and presentation” and “General reporting guidance,” both of which are very broad categories that would have been difficult to assess, particularly as their components overlapped with some of the Guiding Principles, like “Materiality.”

Because of materiality's importance, we subdivided it into three Special Factors: (1) whether material risks were identified in the reports, (2) whether there was an explanation of how these risks would be mitigated, and (3) whether the company included a materiality matrix within the report. Two other Special Factors considered were “Stakeholder engagement” and “Connectivity of information” (matching the Consultation Draft). Because of the growing importance of the Internet in corporate reporting, we also assessed the company's website to see if it supported its integrated report. Finally, as an indicator of whether the company regarded sustainability as core to its strategy or merely a program, we examined the letter from the Chairman, CEO, or other senior executive to see whether the commentary described the company's goals and objectives in the broader context of sustainability issues confronting the organization. We considered it to be a reflection of “the tone at the top” and whether sustainability was being embedded into the business strategy or simply viewed as a bolted-on initiative.

The scoring process was based on a scale of 0–3, with 0 the lowest and 3 the highest number of points awarded for each factor. The maximum score per report across all factors is 60 points (20 factors scored at 3 points each). Every reasonable effort was made to ensure that scoring was as objective as possible, but some degree of subjectivity was inevitable. An inter-rater reliability of 80% was established to maximize consistency of scores across analysts. To accomplish this, each analyst scored the same five reports, and scores from each analyst were compared for each factor and evaluated for their inter-rater reliability. Scoring was consistent at least 80% of the time. Reports were distributed across the team to balance the workload and each analyst initially scored each report on the 20 factors. Once all reports had been scored, each analyst reviewed scores again and, in some cases, made adjustments to scores based on individual factors where appropriate. All averages, totals, correlations, charts, and graphs were generated from these data.