Chapter 30
Metrics that Matter
Social media measurement should not be an afterthought left solely to software platforms auto-generating fancy graphs and charts. Done properly it should be tailored to your organization's objectives, with emphasis on what has changed as a result of your work.
Output, outtake and outcomes. Three words that most PR practitioners should be familiar with but to which may not have given much thought. Yet in the understanding of these three simple words lie the challenges and the opportunities that the PR industry faces when looking to measure its effect successfully and credibly in both traditional and social media measurement.
Let's begin with looking at each of them in turn. AMEC, the International Association for the Measurement and Evaluation of Communications, which does so much to promote education and better understanding in communications measurement, has created an online glossary of terms used in PR and social media measurement. AMEC defines each of these terms as:
“The material and activity that the PR professional generates such as a press release, email, events etc. as well as the ensuing media coverage that is generated. Outputs will also include proactive communication by an organisation on its owned media channels and properties.”
“What an audience now understands having been exposed to content about an organisation or a brand. Outtake occurs before an outcome, although some pundits ignore outtake and just discuss outputs and outcomes.”
“Something that has happened as the result of a campaign. In public relations this would typically be defined as a measurable change in awareness, knowledge, attitude, opinion, behaviour or reputation metrics.”
Put simply, outputs are the things and content that we create and distribute as well as the content that others write about us. Outtakes are what our audiences now think having been exposed to our outputs and outcomes are what the audience has done as a result of exposure to our work. Before seeing an output, our audience was likely to have been unaware, once they have seen the content they become aware, and then once aware, they move to the last stage, the desired outcome itself (see Figure 30.1).
In the old world of traditional media relations, media analysis and media evaluation usually only went as far as measuring the outputs – the press cuttings and sometimes the video and audio clips (although more often just the transcripts of these or even a summary alone) – generated from campaigns.
No matter how sophisticated the analysis technique that was used, how clever its scoring system, algorithm or index, it couldn't hope to measure anything beyond the content itself. The effect that content had on its audience's thoughts (outtakes), or the effect that it had on its audience's actions (outcomes) were in all but the fewest cases not measured, the reason for this being normally put down to cost. Cost was an issue because to measure outtakes – changes in thoughts – and outcomes – changes in actions – would have involved costly market research.
Gaining an accurate understanding of the outtakes and outcomes would also have necessitated complex econometric modelling (or market mix modelling) which was beyond most PR budgets or handled in different marketing silos from the communications team.
Here lies the problem. No organization measures its success on output metrics alone. Instead they focus on the metrics that matter – the outcomes. The sales, the voluntary donations, the client retention, the like for like uplifts, the share price, the attendees at an event – all of these things are the numbers that organizations obsess about. Any organization that is succeeding in its outcome metrics isn't going to worry if its outputs aren't quite what they had planned. It's as irrelevant to an organization how many press releases were written or how many articles ended up in print as whether the staff turned up to work a minute or two late or took an extra lunch hour. What really matters to all organizations isn't the outputs but the outcomes – what has happened as a result of all of the organization's efforts. If a sales team is smashing its targets does it matter that they arrive occasionally a few minutes late for work?
Organizations focus on outcome metrics, and so too should the PR executive in today's world of traditional and social media communications. Many PRs, when confronted with the challenge of measuring social media, sign up to an online listening tool in the mistaken belief that running some pretty charts from it counts as measurement.
There are a couple of problems with this. One problem is that these charts tend to focus on counting just the “output” metrics from the social media content and are not tailored to any organization's objectives. Facebook likes, Twitter retweets, follows, number of pieces of content, etc. are all too often positioned as measurement when in fact they may not be telling you anything meaningful at all.
The other problem is that many of these tools have incorporated their own made-up and proprietary scores to try to help identify “influence”, “reach” and “engagement”. None of these scores and indexes agree with each other or provide consistent, accurate or meaningful data. Even the basic metrics that they all try to capture consistently never agree with each other. Even simple things like volume of content differs wildly across providers owing to the different methods that they use to source the content, the accuracy of their spam filters, the quality of their search algorithms and their licensing agreements with the different social media channels.
We need to be wary of focusing on counting what is easy to count, rather than measuring what matters. The good news is that to move beyond the basic metrics, to measuring what matters, is to follow the process that all high-performing communications teams have always done.
Start with your organization's current goals and objectives. From here map your PR team's social media goals and craft measurable objectives to support those of your organization. Remember that all objectives must be “SMART” – Specific, Measurable, Achievable, Realistic and Time bound. Focusing on supporting your organization's outcome objectives will ensure that you drive the right behaviours in your social media efforts.
Every time you establish an objective, ask yourself up to three times why it matters to you. For example, if you find yourself setting an objective of increasing retweets, ask yourself why this matters. What is it that you really want from retweets? What will they achieve? More awareness? Better exposure? Better understanding of one of your organization's messages? And if so, amongst which particular audience? Asking yourself why a number of times will really help you to understand your true objectives and how best to measure them.
Another useful technique to focus on to bring credible metrics to your social media work is to map what you're doing against the well-known marketing funnel. In Figure 30.2 I have listed on the right the familiar stages of the funnel that organizations have been using for decades. To the left I have listed how the funnel might look from a social media perspective.
The challenge now is to think about each of these stages from your organization's perspective and shape some relevant metrics into each.
A non-exhaustive list of suggestions could include:
OTS / impressions / eyeballs
Brand mentions
Share of conversation
Messages
Tone
Search rank
Click-throughs
Comments/posts ratio
Number of links
@ mentions / RTs / RT %
Bookmarks / likes / votes
Pins
Shares and Likes
Subscribers
URL visits
Awareness
Resolution rate
Ratings
Reviews
Likelihood to recommend
Net promoter score
Footfall
Purchase / donations
Website visits and downloads
Coupon redemption
Endorsement
A non-exhaustive list of potential metrics that could be used to measure each stage of the process. Tailoring these to your objectives is crucial.
What you will see is that as we work our way through the different stages, the metrics by and large change from output metrics through to outtake metrics and finally the metrics that all organizations want to see – the outcome metrics.
The important thing to do in each case is to think about these stages from the perspective of your own campaign. Not all of the metrics suggested here will be relevant to you and there may be others that you should include that are not in the list. By working through your objectives, and fitting them against each of the social media funnel stages, you should be able to come up with a tailored list that is appropriate for the needs of your organization.
For additional help, support and ideas on how you can frame relevant metrics to the work that your organization is undertaking, refer to AMEC's website. Suggested scorecards and frameworks are all available for download.262
A single number to measure social media does not, and will never, exist.
This is the key point about social media measurement – there isn't and won't ever be a single metric that can measure your effect in the space (as frustrating as that might be). The way to measure with success is to design your own suite of relevant and appropriate metrics that are shaped to each campaign's objectives. And if these objectives are aligned to your organization's objectives and KPIs then you will be in a great place to really prove the value of your work.
To summarize, here are some tips and key points to remember when measuring social media.
Richard Bagnall MCIPR (@richardbagnall) ran Metrica for 15 years before selling it to Exponent Private Equity, who merged it with Durrants and Gorkana to form the Gorkana Group. After three and a half years, with Gorkana Group integrating the evaluation business and building it to become one of the world's largest specialist communications measurement companies, Richard left to pursue other opportunities within the industry. Richard chairs AMEC's social media measurement group, is a member of the CIPR social media panel and is also a member of the Cabinet Office's Evaluation Council.
Note
262AMEC's social media measurement resource page: http://cipr.co/WLaTTX
3.15.168.214