Make ROI From Data Analytics a Reality
Is Your Analytics Initiative Worth the Effort?
Have you ever been part of a team building highly complex data solutions that no one seems to understand or value? Do you have no idea what business problem you are solving with your so-called data “solution”? Is your team working without a clear vision or priorities?
There is no use building anything when its value cannot be measured or understood. The pandemic has forced many organizations to adapt quickly to new ways of functioning, like remote work, and there is an urgent need to better understand how to use data insights to sustain business. But there is a disconnect between companies’ desire to use data insights and their ability to take action. If people do not know what to do with this insight, it adds no value. And if no action is taken, it adds no benefit and thereby provides no monetary nor nonmonetary value. In other words, the data adds up to a whole lot of nothing. Because hiring data team members is not cheap, it is vital to ensure that your data initiatives benefit you and that you are able to track that benefit.
With all the buzz around data technologies, artificial intelligence, and machine learning, one would assume that all organizations allocate a generous budget to their analytics initiatives. But have you noticed an opposition or reservation among business leaders to allocate more funding for future data initiatives next year, while analytics teams struggle to justify the business value they created? Although there are lots of industry discussions around the need for data, analytics teams often have trouble highlighting the return on investment (ROI)—the business value—of their work.
Your average analytics team will have no problem highlighting, say, the five biggest technical improvements they made in the past year. But ask the same team to explain how these changes impacted business outcomes, and watch them go blank. It’s tough to measure the ROI of data analytics projects, to quantify the benefits. Often the ROI for analytics project is subjective in nature, and tough to tie to the bottom line. Drawing a line between money spent on data product development and business outcome or revenue is not an easy task and a skill—unfortunately, a rare one.
Before continuing further, lets answer what exactly is ROI. Return on investment (ROI) is not a concept unique to data projects; it applies to any industry. In simple terms, ROI is a determination of the benefit of money and effort spent.
With advancements in data technology, the sky’s the limit when it comes to improving data solutions. So it can be a quest to figure out how much analytics is enough for a particular team or organization. Measurements of ROI can help with that. ROI can also help with decision making on the What?, How?, and How Much? analytics is enough for a given problem.62 It can help answer questions like can we afford this initiative? How much time does it take to get a payback? How feasible is it to implement this data initiative enterprisewide?
It’s easy to get excited about the ROI a data initiative could produce, but it is equally important to check if the solution is feasible or even realistic. Some of you might be familiar with the “Netflix Prize,” which was announced by Netflix in 2006.63 It was an open competition inviting participants to improve their recommendation algorithm by 10 percent. By 2009, Netflix did award $1 million to a team, but it decided never to use the winning algorithm. The reasoning was twofold. First, Netflix could not justify the engineering effort required to implement this algorithm in production and second, between 2006 and 2009, Netflix’s business model changed direction: from DVD rentals to streaming services. It is not enough just to be able to calculate the ROI of a data project within its own narrow silo. It is essential to constantly evaluate need, market conditions, and feasibility. Unless it is a very small organization, your company will most likely be working on more than one data project—and those other data initiatives may well influence the ROI of the one you are assessing.
For a data analytics project, value can come in many forms, including quantifiable or tangible ROI and not easily quantifiable or intangible ROI. Examples of tangible ROI are an increase in number of customers, or reduced hours to add a product feature that can be tied directly to an increase in revenue. Examples of intangible ROI are a strategic direction to enter a new market, or database design changes that cannot be tied directly to an increase in revenue. At surface level it appears as a cost center than revenue to develop these changes. Sometimes we need to be patient or dig deeper to recognize the underlying value on intangible ROI. In other words, entering a new market may increase market share or speed brand awareness prior to competitors while database design changes are needed to protect against potential security threats and improve efficiencies, which may go unnoticed.
In simple terms, tangible ROI is about numbers like increase in revenue and decrease in cost. Intangible ROI is a qualitative, subjective, or “soft” measure, which is challenging to measure. There is a human element to anything we do in technology that is tough to measure. Increased brand reputation, customer retention, and happy team members are all examples of intangible ROI. Think of intangible ROI in terms of the cost of missing out. The $4,000 I spend on a vacation to Paris for four nights is tangible. If I want to save $1,000, I can stay three nights instead. But the happiness I get from my trip results in the intangible benefit of greater productivity after I return. Understandably, intangible ROI is more difficult to measure than tangible ROI, but that does not mean it is less crucial a driver of success.
An organization will move through different phases when it comes to ROI calculation. Data ROI mindsets evolve over time and can vary widely between organizations. Here are some examples of different organizational mindsets when it comes to data ROI:
• Undeveloped ROI Mindset (No ROIs of the world): Your technical teams build data solutions and implement the newest technologies because they are fun to work on. “Since everyone else is implementing cool new data things, we need to as well.” This mindset does not think in terms of the business value these data initiatives bring. Teams start their initiatives with data solutions instead of first clarifying their business problems and questions. Under the leadership of this mindset, teams build new tools, but these fail to be widely used because everyone is more comfortable with legacy system and there hasn’t been enough training to explain the key functionality of the product.
• Curious ROI Mindset (Somewhat ROIs kind): This mindset understands the need for ROI from data initiatives but does not know where to start or how to quantify. It attempts to solve all business questions by treating all questions equally—instead of focusing on the most valuable questions first or trying to determine which effort will lead to “largest slice of the cake” returns. Hence available data team capacity is not efficiently used.
• Developing ROI Mindset (Getting to ROIs kind): Organizations following this mindset understand the need for ROI in data projects and have some guidelines to calculate ROI for any new intakes or requests. They have some ROI calculator user interfaces (UIs) available, but they are not easy to use, and people avoid using ROI calculators unless they are pushed to do so. Instead of performing ROI calculation consistently, they pick and choose as needed. This mindset often has a better grasp of ROI when projects are small and involve minimal effort to develop.
• Advanced ROI Mindset (Efficient ROIs kind): Organizations with an efficient data ROI mindset embed ROI in their work culture, and everyone appreciates the value it brings. Although not every data team member performs ROI calculations, the team has clear workflow process to vet different data requests and prioritize based on factors like ROI and feasibility. They also have the necessary tools and communication channels to collect important data and demonstrate progress toward returns from their data initiatives. They constantly check market conditions, collect user feedback, and adapt to create the most valuable solutions that can benefit the company.
Tangible ROI
There are various methods available to calculate ROI in terms of net income, annualized income, and factors such as inflation. Some elements of ROI calculation are economic and finance concepts that lie outside the scope of this book. Here the focus is on the questions and data required to determine ROI.
I will walk through two lightweight examples to explain tangible ROI. They are simple, but once the concepts are clear, you can apply the same logic to any complex solution you are building.
Example 1: New Patient Portal
Your pharmacy has a legacy patient portal that crashes frequently and cannot integrate with other systems or application programming interfaces (APIs). In addition, the legacy portal cannot be accessed from a mobile device. You are building a new patient portal to replace the old system. Although everyone agrees that a new portal is needed, there is no enterprisewide understanding of effort (time and cost required to build a portal) to value realization (benefits like more patients, less customer service calls) the new portal will deliver. You are tasked with coming up with ways to determine tangible ROI.
To understand the ROI new portal can bring in, let’s first define what success means for this new system. Many times, ROI becomes a tough question to answer as we are in a rush to get started with building a new fancy data solution than answering fundamental question of what the new system will do better than legacy (if it exists).
Does your portal’s success mean any of the following?
• More new patients signing up to the portal.
• Patients accessing new features of the portal, like a find-a-doctor tool.
• Patients logging in more frequently.
• Do patients use portal to order prescription refills.
• Patients finding information more easily, thereby reducing customer service calls.
To evaluate the performance of the new system, it is critical to establish baseline data to compare new values to old values. “Analytics Design Sheet” (ADS) template could come in handy for this kind of scenario.64 ADS with its four quadrants such as context, decision, data, and analytics can help facilitate brainstorming the kind of analytics solution we are building and the kind of data needed to evaluate the ROI.
By tailoring an ADS template to include ROI, we can define Example 1 as follows:
• Context/problem statement—Build a new patient portal to replace a crashing legacy system, with additional features like ability to order prescriptions to overcome crashing legacy system.
• Decision—Will we roll out a trial version to a sample group of patients to determine usage, do we have idea of estimated effort needed, and the feasibility of seamlessly migrating a high volume of legacy patients to the new portal.
• Data—Identify and collect usage data for the legacy system, customer service history.
• Analytics—Use usage trends, predictive models to forecast revenue increases from higher patient enrollment, a reduction in calls to customer services, and ROI analysis.
Example 2: Inventory Management Suite
You own a chain of coffee shops, but each store operates individually, ordering its own inventory, with some shared procurement contracts for coffee beans and milk, along with ad hoc purchasing. Although high volumes are ordered across stores, still you are unable to establish better procurement contracts. In addition, stores are unable to share their inventory even with nearby sister stores. To improve efficiencies, your company is building a centralized inventory management suite, which will be used by the chain of stores to predict demand and order different varieties of coffee on time.
Let’s start by defining success for the coffee inventory management system:
• Able to see holistic view of inventory quantity and inventory value $ across all the coffee stores.
• Showing all pending and completed orders.
• Calculating required lead times for supplies by vendor.
• Comparing contracted price with purchase order price.
• Sharing unused inventory among stores according to demand.
Using the four ADS quadrants, we can define Example 2 as follows:
• Context/problem statement—Because there is no inventory visibility across stores, build a centralized inventory management system.
• Decision—Will you roll out prototype procurement contracts for a group of a few stores first, and compare the results against ad hoc store purchases.
• Data—Since most orders are managed in a variety of ways—paper receipts, Excel spreadsheets, and phone orders—consider performing data entry for 12 months to create a data baseline.
• Analytics—Forecast demand based on historical store
traffic in order to stock adequate inventory, reduce waste, procure better pricing for raw materials, and provide training sufficient to ensure that staff are comfortable using the new system.
These are both straightforward examples of calculating ROI: consider the cost of building a patient portal or inventory management system; calculate the approximate savings, like the decrease in customer center calls or the savings in procuring coffee story inventory; and use these numbers to estimate how long it will take your organization to break even and earn a profit. This kind of ROI calculation will help justify building in house analytics product or use commercial off-the-shelf product to cater to the needs of the company.
Intangible ROI
When calculating ROI, it is appealing to stick with tangible metrics and to ignore hard-to-measure intangible ROI. But by doing so, you are missing an opportunity to see the complete picture of your products and to improve ROI. In the age of social media and digitization, there is a lot of information that can help you understand how your product and brand are perceived by customers. Let’s revisit examples 1 and 2 again in terms of intangible ROI.
Example 1: New Patient Portal
Let’s add few additional considerations to our scenario of a patient portal.
There is lot of discussion on your company website and social media about the issues patients are facing when they use the new patient portal. People are frustrated by the number of steps required to set up their profile. Also, while it looks like new patient enrollment is going up, there is data-quality issue that is unable to do patient match efficiently, thereby creating several duplicate patients even when there is typo in name or other attributes. Another new feature introduced was a module for specialty patients who need training to use some drug delivery devices and need to connect with nurses. This is well received with patients able to schedule appointments with nurse immediately after receiving the devices. This achieves a strategic business objective to address changing market needs by introducing a first-in-kind ahead of the competition, and it’s another intangible ROI for the patient portal, which is driving competitive advantage.
Example 2: Inventory Management Suite
For this scenario, let’s add the human element to your coffee stores. After introducing the new inventory management suite, resignations among store managers have decreased. New stores have expressed interest in learning the new system and have asked if their store can be the next to be migrated. The customer experience in stores with new system has improved because staff are not mired in lengthy procurement processes over the phone. Overall employee retention, customer experience, and job satisfaction have improved. There are also stronger partnerships between stores, and a dose of healthy competition to spur performance. These are key intangible elements that indirectly affect the bottom line.
Using Key Performance Indicators (KPIs) to Determine Value
Organizations implement data projects for a wide range of reasons, and some use cases are more common in certain industries than others. For a banking institution, fraud detection may be the priority, while a pharma company might prioritize increasing patient access to drug and patient experience. For other organizations, reducing cost is more of a priority than spurring innovation or improving market share. Sometimes use case is regulatory or risk mitigation by nature, which focuses on answering “What is the cost in terms of fine for not doing it.”
Is KPI and ROI same? What exactly is KPI and why do I need it? As the name suggests, KPIs help in calculating ROI. For the example of our coffee store, one KPI is the lead time required to get ingredients; another KPI is the monthly cost of procuring ingredients. These individual KPIs paint a part of the picture such as lead times are decreasing and procurement costs are increasing. But they do not show the whole picture that you need to justify a new inventory management suite. A calculation of ROI allows you to compare the cost of the suite (costs $X to build it) against its consequences—positive and negative—for the coffee chain. And it is impossible to calculate ROI without good and accurate KPIs.
Here are some KPIs for the most common use cases:
• Webpage views, downloads (actions taken by users on your webpage).
• New subscribers (Are more people signing up?).
• System downtime (compared to the downtime of the legacy system).
• Number of defects (Is the new system stable?).
• Procurement lead time (Do you manage your inventory efficiently to order before you run out of material?).
• Purchase order accuracy (compared to invoice).
• Rx refills (Are patients refilling their drugs and adhering to therapy?).
• Number of new patients (Are new patients starting to take the medication?).
• Overall satisfaction (of patients, customers, and employees).
• Readmission rate (patients returning to hospital with same conditions they originally came for).
• Churn rate (how often customers quit your product).
• Acquisition costs (acquiring customers may cost more than retaining them).
• Fraud to sale ($ amount lost in fraud to $ amount gained in sales).
• Medical claim frauds (frauds per person or per medical office).
• Sentiment score (sentiment measured through, e.g., customer service e-mails, chats).
Conclusion
An organization may have a variety of new ideas, but it cannot work on all of them. There needs to be a plan to focus data team capacity on what matters the most. Because the data space is constantly changing, it is always tempting to try out the next cool thing. Although these feelings are natural in technology work, no company can sustain its data approach without a road map of which data initiatives bring the most desired outcomes and the ROI for doing it or the risk of not doing it.
• Start with a business problem instead of a data solution. With this approach, many times businesses realize they do not even have the correct data to answer those questions, thereby eliminate going directionless and solving irrelevant problems with existing data.
• In addition, it is critical to communicate a clear timeframe for the realization of ROI expectations. Leadership may want to see value within the first month, when a prediction of 10 months is more realistic. Measuring ROI is simpler for smaller projects, but complexity grows with larger initiatives. Within your organization, there should be alignment not just about the value of data initiatives but also about the time required to realize that value. More often than teams realize, this is overlooked, leading to disappointments between business and technical teams.
• Every business is unique with different kind of problems and use cases for data initiatives. So, it is worthwhile to classify data initiatives—into categories such as strategic direction or operational excellence—in order to define ROI creation based on such use cases.
Creating awareness of the need for ROI for data initiatives is one part of the puzzle. Giving your teams the required tools to determine ROI is the other part. Once your organization has established some high-level KPIs for the most common use cases, it is critical to build a ROI calculator that is easy to use. You cannot ask people to measure returns without having the metrics data, tools to track and calculate returns, and some standard communication channel to highlight returns.
Your team will also need subjective intangible ROI to answer questions such as “What is our ROI for this data initiative?,” “Why should we keep funding this when we have no idea of our current benefits?,” and “Why we are building the next generation of this data product when almost no one has adopted the existing version?” It could be web portal with a good self-explanatory user experience to determine high-level ROI, or even an Excel spreadsheet that can be improved over time. At times, there are similar driven data solutions built by various technical teams just to sunset them without realizing any value. To avoid such scenarios, your organization needs a value-driven or value proposition mindset. A value proposition mindset is created when everyone in the company asks questions like “Is the feature I want to build worth the effort?” and “Do other similar solutions already exist within the organization?” Not everyone will be able to answer these questions, but this mindset will encourage ROI from your data projects a reality.
Before picking your next data initiative, focus on the ROI it may generate. And consider different aspects in addition to ROI that could influence the decision. Factors such as gaining competitive advantage, being first of the breed, strategic shift of how data is perceived as an organization, regulatory needs, rebranding, achieving operational excellence, innovation, time to market, feasibility, market conditions, and team efficiency, all will influence toward driving ROI in different intensities and in turn affect Data and Analytics project prioritization.