Chapter 13
Law 9: Drive Customer Success through Hard Metrics
Author: Jon Herstein, Senior Vice President of Customer Success, Box
Relevance
Executive Summary
Customer success is still relatively new as a formal organization within the enterprise. As with any new business endeavor, maturity is required to ensure long-term viability. It's that time for customer success. Repeatability, process definition, measurement, and optimization are the hallmarks of that maturation. We see glimpses of these at more mature recurring revenue businesses, but there's still a long way to go for most.
Ultimately, the purpose of customer success, like any other part of a thriving company, is to achieve real business outcomes. Defining what success means both for you and for your customers, then establishing clear metrics that will deliver those business outcomes, is a necessary part of accelerating the maturation process. You can't improve what you don't measure.
In the late 1980s, Carnegie Mellon's Software Engineering Institute began to develop a process maturity framework that would help organizations improve their software process. Published several years later, the Capability Maturity Model for Software (CMM) became the go-to reference for assessing the process maturity of software development organizations. Equally important, the CMM framework has been used as a more generalized reference for determining how mature an organization—and its processes—has become, and serves as a guidebook for how to move from the initial level of maturity to the optimizing level.
Why is CMM relevant almost 30 years later in the (very different) world of customer success? Its basic premise is that, as organizations evolve their capabilities, they get progressively better and more predictable at executing their mission. This is true whether that mission is building great software or ensuring a consistently excellent customer experience. And the hallmarks of progression through the levels of maturity are repeatability, process definition, measurement, and optimization. Putting it all together, if we can measure and optimize the processes relevant to our customer success organizations, we are significantly more likely to reach our likely business objectives (high customer satisfaction, low churn, revenue expansion, etc.).
At Level 1 (initial), work gets done through the heroic actions of committed people without much regard for process or repeatability. Sound familiar? If you manage a handful of CSMs (or fewer), this is probably your daily reality. The objective your CSMs are focused on is along the lines of “Do whatever it takes to make your customers successful and make sure they renew!” The role of the CSM at this stage is likely to be poorly defined; details beyond this objective are left to individuals to figure out as they go. Assuming that you've got good people, this will actually work for a while. But your short-term gain (customer happiness) will likely result in long-term pain (overworked people, inconsistent delivery, uneven/inconsistent results).
Progressing to Level 2 (repeatable) occurs when the necessary process discipline is in place to repeat earlier successes. From there, achieving Level 3 maturity (defined) occurs when the process is documented, standardized, and integrated into a standard process for the organization. At this point, the fundamentals of a repeatable process methodology are in place, and what remains is measurement (Level 4: managed) and consistent improvement (Level 5: optimizing).
Assuming that your customer success organization has identified its repeatable processes and clearly defined and documented them, your focus will turn to active measurement and optimization. But what can and should be measured, and what are the benefits of doing so? Broadly speaking, you can think of three categories of metrics to explore: (1) customer behavior, (2) CSM activity, and (3) business outcomes. You will find a vast set of possible metrics within each of the buckets, and the following discussion will attempt to provide just a few relevant examples. Each business (and accompanying customer success organization) will need to determine which of these metrics matter and how, exactly, to define and measure them.
Customer and User Behavior
One of the biggest advantages of the SaaS delivery model for software, as compared to on-premise software, is that we can instrument and measure every aspect of how customers use our products. Previously, a software vendor had no practical way to determine whether or how the intended user base used its software. In properly instrumented SaaS applications, we are aware of every login, click, upload, download, error generated, and so forth. We know the frequency with which users perform specified activities. And depending on the nature of the product, we may also know the business value of these activities (e.g., a SaaS provider of an e-commerce platform will know the value of the transactions it has processed). The trick, of course, is correlating usage metrics to derived business value (for the customer) and how that will ultimately affect retention/expansion.
Examples of user-based metrics may include (but aren't limited to):
- NPS
- Logins and logouts
- Usage of specific product features/platforms (online, mobile, API)
If you are operating a B2B model, you may also aggregate user-level behavior (and other customer-level behavior, such as payments made) into a higher-level view of customer “health.” This may include identifying risk factors that you have correlated to likelihood of churn, such as payment/nonpayment, engagement with the customer's administrator, and referenceability.
One important note of caution: The behavior of your customer's users serves only as a proxy for the business value the customer is deriving. To paraphrase Nick Mehta, Gainsight CEO, no one buys your software so they can log into it. Your customer has subscribed to your solution to fulfill one or more business objectives: find more leads, generate more revenue, make manufacturing more efficient, or enhance collaboration with suppliers. The key is to understand what those objectives are and how your product relates to them. In some cases, you won't be able to ensure your customer is meeting its intended objectives solely through the instrumentation built into your product. For example, if customers are using your file-sharing solution to eliminate FTP servers, you'll have to ask them whether they've actually done so; your product provides no visibility into this. Spend time with your customers at the beginning of the relationship to understand their business objectives and agree on how you will jointly measure the results.
Customer Success Manager Activity
Once you've defined the processes for your CSMs, it is natural to wonder how well those processes are being followed. From there, you will want to understand how the activities in which CSMs are engaging (or not engaging) affect customer sentiment and retention. Having a comprehensive understanding and proper measurement of these processes will lead to insights into the performance of your people, as well as into how much these activities actually matter to business outcomes. For example, are your QBRs as effective as you think they are in driving great adoption of your product? Do face-to-face visits outperform e-mail and phone calls when it comes to customer satisfaction?
Examples of CSM activity metrics may include:
- Frequency of various types of interactions with customers (QBRs, e-mail updates, phone calls)
- Support ticket volume handled by CSMs (rather than your support team)
- Timeliness of risk identification
- Effectiveness of risk mitigation efforts
Business Outcomes
An added benefit of maturation toward measurement and optimization is greater predictability of outcomes. Want to know how many customers a CSM can effectively manage (ideal account ratio)? Measure the relevant business outcomes for cohorts of CSMs with varied account loads. Interested in understanding how effective formal quarterly business reviews are vis-à-vis more frequent, informal check-ins? Measure customer engagement and satisfaction for cohorts that vary on this dimension.
Note that this section refers to the business outcomes that matter to you (retention, expansion, etc.). You must work with various functions within your organization (product, marketing, sales, finance) to determine what “success” looks like and which metrics indicate how you're doing. You'll be designing your processes, activities, and metrics around the definition of success and measuring accordingly. In many cases, you will share responsibility for success with the other groups: For example, customer success and product are jointly responsible for ensuring that users of your product adopt it. The more clarity you can drive around ownership of these metrics, the more you can refine the processes and behaviors your team will execute.
Examples of business outcome metrics may include:
- Gross retention
- Net retention
- Expansion
- Logo retention
- Customer satisfaction
- NPS
Clearly defining what success means, both to you and your customers, ensures greater clarity of your customer success team's mission and responsibility. Once you've achieved alignment on this definition, it's critical to articulate the things you will measure to demonstrate how the team is performing. These metrics enable customer success leaders to prove the value of the customer success organization and improve your contribution to the company's overall performance over time. Finally, your CSMs will thank you for the clarity of purpose this brings them, as well as the enhanced ability to truly understand their own performance and contributions.
Remember: you get what you measure! So figure out what matters, and then start defining and focusing on those key metrics.
Additional Commentary
This law is obviously directed at those who have actual customer success teams. If that's the case for you, it's absolutely necessary to actively manage your team with very specific metrics, just like you do your sales team or any other team. At some point, headcount needs to be justified by metrics, not by begging. If you are a volume B2B business or a B2C business, you will someday, if you don't already, have a customer success team. It might be only one or two people for thousands, or even millions, of customers, but someone will be responsible for the customer experience and will be measured on the key customer success metrics. The techniques of a team of CSMs who have some reasonable number of accounts each (5 to 150) will clearly be very different from those who are responsible for thousands or millions. We've explored that at some length with our discussion of high-touch, low-touch, and tech-touch models. At the highest level, the driving metrics are the same for all models, and they are basically the company-level metrics, too—retention, churn, upsell, and so on. If you are managing individuals with CSM responsibilities, you'll need to go deeper on their metrics. Retention, upsell, and churn are the right long-term metrics, but they are lagging metrics and not particularly predictive. More on that in the high-touch section that follows.
In the infancy stage of customer success, which we are still in, the measurements really have been akin to baby food. We've had hundreds, even thousands, of very fluffy, one-on-one conversations with our CSMs. We ask good questions wanting to help them, such as:
- How are your customers doing overall?
- Are there any customers at risk of churn?
- You've been working with customer X on some challenges for the past 60 days, are we making progress?
- Customer Y churned. What could we have done differently?
- How can I help?
These are all reasonable questions to ask someone responsible for the retention and overall success of a particular book of business, but none of them are very measurable. This is again a place where a customer success management solution can be extremely helpful. They have the ability to change your one-on-ones so they sound more like this:
- Your average health score across all your customers is six points lower than the rest of the team. It looks like the component that is dragging you down is the executive relationship. Let's put together a plan to change that, starting with the lowest-scoring customer.
- You have three at-risk customers up for renewal in the next 90 days. Let's review your action plan for each of those customers.
- Your upsell rates are 10 percent higher than the next-best CSM. That's awesome, and I'd like you to put together three slides we can review and you can share at our next team meeting to help everyone else step up their game to your level.
It's not hard to figure out which one of those one-on-ones is more effective for both the manager and the employee. As with all disciplines, active management is possible only if there are clear measurables. This allows you to manage the results but coach the process. It's imperative, as the discipline of customer success matures, that the ability to effectively manage the team and the individuals evolves as well and is based on specific measurables that drive bottom-line business value.
High Touch
Managing and measuring customer success people in a high-touch model is very similar to managing sales reps. For a sales rep, there's only one metric that really matters, right? How much did you sell? At some point, that's the lone measurement that determines success or failure. But, does a good sales VP wait 12 months, or even a quarter, to see what a sales rep's results are to determine whether they are good? Of course not. There are lots of things they'll be looking at along the way that they believe are indicators of future success. Some of the more measurable ones are
- Pipeline size
- Pipeline growth
- Pipeline movement
- Number of calls
- Number of meetings
- Number of proposals created and sent
- Average forecasted deal size
And there are many more. Of course there are also a bunch of more subjective things that will be observed and coached, too, such as skill at delivering the standard sales pitch, ability to overcome objections, and so on. Every job has some highly measurable aspects and some less tangible elements, too.
Customer success is no different. I'll argue that the key metric to determine the quality of an individual CSM or the whole team is net retention. This takes into account both retention and upsell. I've said this before and I'll say it again. Successful customers do two things: (1) They stay with you as a customer (renew their contracts if on a subscription). (2) They buy more stuff from you. If customer success's job is to make customers successful and that's what successful customers do, then net retention is the metric that matters for them. However, just as with sales reps, you probably don't want to wait for 12 months to see the net retention number for a CSM's entire book of business. You'll want to measure the elements that help you predict whether the CSM is successful far in advance of a renewal or upsell event. Much like a CRM system does for sales, a customer success management solution can do this for customer success. You can also track many of these manually if you don't have a CSM solution:
- Health score across book of business
- Health score trends
- Level of direct CSM engagement
- Number of triggered actions (low survey score, no product usage)
- Number of triggered actions completed
- Number of upsell opportunities identified
- Number of positive relationship activities (references, case studies, etc.)
And, of course, there are subjective things to watch, too, such as their depth of product knowledge, ability to utilize other resources wisely, and many more.
The bottom line is that there's no excuse for fluffy one-on-ones any longer with high-touch customer success teams. We need to rapidly move into highly measurable, highly actionable metrics that help the individuals to improve and drive positive company results.
Low Touch
Everything we said with regard to the high-touch model is true here in the low-touch model, too. The same challenges and opportunities exist to start measuring the things that matter with regard to building an effective team or program and, more importantly, having a positive impact on your customers.
In many ways, this is more critical the lower touch your model is, because you can't rely on relationships to win the day or understand the customer. If I'm a high-touch CSM with five clients, I can tell you with a great deal of accuracy the health score for each of my customers. I just know because I talk to them all the time. If I have 200 customers and I'm forced to operate with a much lower touch, I will be much more dependent on any kind of metrics that can be automatically reported about my customers. In this environment, it becomes almost imperative to have a CSM solution, but many of the following could be tracked manually if an automated solution is not available. The list below is an extension of the list we started earlier in the high-touch section:
- Survey scores (this is often incorporated into the health score but can be tracked separately, especially if you haven't developed a health score yet)
- E-mail engagement (what do customers do with e-mails that come directly from you or from your marketing team?)
- Number of support tickets opened (this may not be a good measurement of a customer success individual or team as it's probably out of their control, but it can shine the light on troubled customers)
- Invoices—happy customers tend to pay their bills on time
Because you're ultimately measuring your customer success team on how well their customers are doing, anything that helps you understand that about your customers is an indicator (to varying degrees) of the success of the individuals responsible for those customers. It's not feasible to track everything, but track what you can so you have a temperature on your customers and some insight into how your team is doing.
Tech Touch
The great news about the tech-touch environment is that there are volumes of customers, which makes experimentation easier. In this model in which virtually all of your interactions with customers are technology driven, with e-mail being a primary vehicle, you can easily do some A/B testing to determine what works best. Let's say that you have an intervention scheduled for every customer when they reach an annual anniversary as a customer of yours. You could construct an e-mail with the exact same content but two different subject lines, one that says, “Congratulations” and another that says “Happy Birthday.” And then you just watch the open/bounce/unsubscribe rates to see which was more effective.
In many ways, the tech-touch customer success team operates much like a marketing team whose interactions are primarily digital, too (website, e-mail, webinars, etc.). This leads to the conclusion that they can be measured in similar ways. That's essentially true. Much like marketing, your tech-touch customer success team should be measured on the effectiveness of their touches. That means measuring things such as:
- E-mail engagement
- Webinar attendance
- Community engagement
- User group participation
For marketing teams, the ultimate measure of their success is leads created (i.e., pipeline). For customer success teams, the ultimate measure is health score. It's not a coincidence that customer health score for customer success teams is analogous to pipeline for sales teams. Simply put, a sales pipeline is a predictor of future behavior, such as likelihood of closing, timing of closing, deal size, and so on. The pipeline is the primary input to a sales VP's forecast. Health score for a customer success VP provides the same insights. An accurate health score is a great predictor of future customer behavior, such as likelihood of retention, possibility of upsell, level of risk, and so on. In the end, everything your customer success team does should be designed to create loyalty, and loyalty is measured in the long term by net retention and in the short term by health score.
The discipline of customer success is changing rapidly, like a child going through a growth spurt. One of the areas of most rapid change is in the measurement and management of our customers and, by extension, the teams that are responsible for those customers. To become an adult with a seat at the big table, this trend needs to continue so that CFOs and CEOs can see the fruits of their investments.